[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240503 08:03:27.777452 12312 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.6.62:36233
I20240503 08:03:27.778862 12312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 08:03:27.779989 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:27.799296 12318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:27.800462 12319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:27.801674 12312 server_base.cc:1034] running on GCE node
W20240503 08:03:27.801801 12321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:28.933118 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:28.933384 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:28.933593 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723408933567 us; error 0 us; skew 500 ppm
I20240503 08:03:28.934427 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:28.945509 12312 webserver.cc:460] Webserver started at http://127.12.6.62:44615/ using document root <none> and password file <none>
I20240503 08:03:28.946445 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:28.946677 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:28.947176 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:28.953541 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/master-0-root/instance:
uuid: "7d4c9555562445cd8fefee7c761c3bbc"
format_stamp: "Formatted at 2024-05-03 08:03:28 on dist-test-slave-6qcz"
I20240503 08:03:28.962122 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.004s
I20240503 08:03:28.967772 12327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:28.968933 12312 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20240503 08:03:28.969290 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/master-0-root
uuid: "7d4c9555562445cd8fefee7c761c3bbc"
format_stamp: "Formatted at 2024-05-03 08:03:28 on dist-test-slave-6qcz"
I20240503 08:03:28.969640 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:29.033313 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:29.034847 12312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 08:03:29.035326 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:29.117090 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.62:36233
I20240503 08:03:29.117189 12378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.62:36233 every 8 connection(s)
I20240503 08:03:29.124897 12379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:29.150635 12379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc: Bootstrap starting.
I20240503 08:03:29.156828 12379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:29.158697 12379 log.cc:824] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc: Log is configured to *not* fsync() on all Append() calls
I20240503 08:03:29.163872 12379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc: No bootstrap required, opened a new log
I20240503 08:03:29.188226 12379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4c9555562445cd8fefee7c761c3bbc" member_type: VOTER }
I20240503 08:03:29.188815 12379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:29.189093 12379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4c9555562445cd8fefee7c761c3bbc, State: Initialized, Role: FOLLOWER
I20240503 08:03:29.189888 12379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4c9555562445cd8fefee7c761c3bbc" member_type: VOTER }
I20240503 08:03:29.190382 12379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 08:03:29.190667 12379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 08:03:29.191004 12379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:29.195173 12379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4c9555562445cd8fefee7c761c3bbc" member_type: VOTER }
I20240503 08:03:29.196048 12379 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4c9555562445cd8fefee7c761c3bbc; no voters:
I20240503 08:03:29.198093 12379 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 08:03:29.198477 12382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:29.201331 12382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 1 LEADER]: Becoming Leader. State: Replica: 7d4c9555562445cd8fefee7c761c3bbc, State: Running, Role: LEADER
I20240503 08:03:29.202509 12382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4c9555562445cd8fefee7c761c3bbc" member_type: VOTER }
I20240503 08:03:29.203238 12379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [sys.catalog]: configured and running, proceeding with master startup.
I20240503 08:03:29.215561 12384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4c9555562445cd8fefee7c761c3bbc. Latest consensus state: current_term: 1 leader_uuid: "7d4c9555562445cd8fefee7c761c3bbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4c9555562445cd8fefee7c761c3bbc" member_type: VOTER } }
I20240503 08:03:29.215415 12383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d4c9555562445cd8fefee7c761c3bbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4c9555562445cd8fefee7c761c3bbc" member_type: VOTER } }
I20240503 08:03:29.216215 12384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [sys.catalog]: This master's current role is: LEADER
I20240503 08:03:29.216284 12383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [sys.catalog]: This master's current role is: LEADER
I20240503 08:03:29.220767 12391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 08:03:29.235256 12391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 08:03:29.237088 12312 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240503 08:03:29.252652 12391 catalog_manager.cc:1347] Generated new cluster ID: 4466f11d30884adf86a11ecc5b17661b
I20240503 08:03:29.252977 12391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 08:03:29.290796 12391 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 08:03:29.293181 12391 catalog_manager.cc:1504] Loading token signing keys...
I20240503 08:03:29.312191 12391 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc: Generated new TSK 0
I20240503 08:03:29.313218 12391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 08:03:29.368604 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:29.379010 12401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:29.381319 12402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:29.384109 12404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:29.385793 12312 server_base.cc:1034] running on GCE node
I20240503 08:03:29.386493 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:29.386714 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:29.386886 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723409386868 us; error 0 us; skew 500 ppm
I20240503 08:03:29.387603 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:29.392887 12312 webserver.cc:460] Webserver started at http://127.12.6.1:42649/ using document root <none> and password file <none>
I20240503 08:03:29.393451 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:29.393635 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:29.393911 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:29.395083 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-0-root/instance:
uuid: "49e3300ff51e489889daf9f659d10b13"
format_stamp: "Formatted at 2024-05-03 08:03:29 on dist-test-slave-6qcz"
I20240503 08:03:29.401672 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20240503 08:03:29.406569 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.407517 12312 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20240503 08:03:29.407815 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-0-root
uuid: "49e3300ff51e489889daf9f659d10b13"
format_stamp: "Formatted at 2024-05-03 08:03:29 on dist-test-slave-6qcz"
I20240503 08:03:29.408106 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:29.435485 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:29.437602 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:29.440539 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:03:29.446115 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:03:29.446372 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.446673 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:03:29.446835 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.572476 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.1:46613
I20240503 08:03:29.572568 12471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.1:46613 every 8 connection(s)
I20240503 08:03:29.581223 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:29.598910 12476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:29.605474 12477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:29.610592 12479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:29.616261 12312 server_base.cc:1034] running on GCE node
I20240503 08:03:29.616937 12472 heartbeater.cc:347] Connected to a master server at 127.12.6.62:36233
I20240503 08:03:29.617365 12472 heartbeater.cc:464] Registering TS with master...
I20240503 08:03:29.617331 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:29.617686 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:29.617908 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723409617891 us; error 0 us; skew 500 ppm
I20240503 08:03:29.618471 12472 heartbeater.cc:511] Master 127.12.6.62:36233 requested a full tablet report, sending...
I20240503 08:03:29.618772 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:29.621860 12344 ts_manager.cc:194] Registered new tserver with Master: 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613)
I20240503 08:03:29.623430 12312 webserver.cc:460] Webserver started at http://127.12.6.2:37781/ using document root <none> and password file <none>
I20240503 08:03:29.624163 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:29.624429 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:29.624783 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:29.624845 12344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60108
I20240503 08:03:29.626356 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-1-root/instance:
uuid: "6bbad1351d304f2da4f413e6522c0bd0"
format_stamp: "Formatted at 2024-05-03 08:03:29 on dist-test-slave-6qcz"
I20240503 08:03:29.633766 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.002s
I20240503 08:03:29.639950 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.640957 12312 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20240503 08:03:29.641276 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-1-root
uuid: "6bbad1351d304f2da4f413e6522c0bd0"
format_stamp: "Formatted at 2024-05-03 08:03:29 on dist-test-slave-6qcz"
I20240503 08:03:29.641563 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:29.661688 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:29.663069 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:29.664640 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:03:29.668169 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:03:29.668373 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.668637 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:03:29.668779 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.793915 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.2:32969
I20240503 08:03:29.794113 12546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.2:32969 every 8 connection(s)
I20240503 08:03:29.803787 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:29.816175 12550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:29.821458 12551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:29.822618 12553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:29.823962 12547 heartbeater.cc:347] Connected to a master server at 127.12.6.62:36233
I20240503 08:03:29.824328 12547 heartbeater.cc:464] Registering TS with master...
I20240503 08:03:29.825018 12547 heartbeater.cc:511] Master 127.12.6.62:36233 requested a full tablet report, sending...
I20240503 08:03:29.825479 12312 server_base.cc:1034] running on GCE node
I20240503 08:03:29.826524 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:29.826783 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:29.826911 12344 ts_manager.cc:194] Registered new tserver with Master: 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:29.827003 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723409826985 us; error 0 us; skew 500 ppm
I20240503 08:03:29.827912 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:29.828406 12344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60122
I20240503 08:03:29.831554 12312 webserver.cc:460] Webserver started at http://127.12.6.3:41355/ using document root <none> and password file <none>
I20240503 08:03:29.832207 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:29.832455 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:29.832773 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:29.834141 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-2-root/instance:
uuid: "d318d4aca45244ba9cd5a8cb36cab286"
format_stamp: "Formatted at 2024-05-03 08:03:29 on dist-test-slave-6qcz"
I20240503 08:03:29.840260 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20240503 08:03:29.844672 12558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.845610 12312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20240503 08:03:29.845983 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-2-root
uuid: "d318d4aca45244ba9cd5a8cb36cab286"
format_stamp: "Formatted at 2024-05-03 08:03:29 on dist-test-slave-6qcz"
I20240503 08:03:29.846351 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714723407607891-12312-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:29.879622 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:29.880857 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:29.884155 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:03:29.887801 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:03:29.888003 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:29.888267 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:03:29.888417 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:30.015858 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.3:42523
I20240503 08:03:30.015915 12620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.3:42523 every 8 connection(s)
I20240503 08:03:30.036944 12621 heartbeater.cc:347] Connected to a master server at 127.12.6.62:36233
I20240503 08:03:30.037396 12621 heartbeater.cc:464] Registering TS with master...
I20240503 08:03:30.038970 12621 heartbeater.cc:511] Master 127.12.6.62:36233 requested a full tablet report, sending...
I20240503 08:03:30.041355 12344 ts_manager.cc:194] Registered new tserver with Master: d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:30.042367 12312 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019558143s
I20240503 08:03:30.042862 12344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60130
I20240503 08:03:30.047529 12312 test_util.cc:268] Using random seed: 666792459
I20240503 08:03:30.107951 12344 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60146:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 08:03:30.112730 12344 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 08:03:30.227027 12392 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:03:30.247686 12512 tablet_service.cc:1467] Processing CreateTablet for tablet 3958d27365e7439494639369ef57cd9b (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240503 08:03:30.243595 12511 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce18a893675482c9720556a30c9ed10 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240503 08:03:30.249001 12510 tablet_service.cc:1467] Processing CreateTablet for tablet 772658b6e692478fae18294150f1f32a (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240503 08:03:30.253454 12510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772658b6e692478fae18294150f1f32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.260890 12511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce18a893675482c9720556a30c9ed10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.265129 12437 tablet_service.cc:1467] Processing CreateTablet for tablet 3958d27365e7439494639369ef57cd9b (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240503 08:03:30.261286 12508 tablet_service.cc:1467] Processing CreateTablet for tablet d785e3daf54f4bb7b50647703534ecf0 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240503 08:03:30.269498 12435 tablet_service.cc:1467] Processing CreateTablet for tablet 772658b6e692478fae18294150f1f32a (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240503 08:03:30.262102 12436 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce18a893675482c9720556a30c9ed10 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240503 08:03:30.272728 12437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3958d27365e7439494639369ef57cd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.269134 12509 tablet_service.cc:1467] Processing CreateTablet for tablet bac0059e531d4156b96e978e5e610408 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240503 08:03:30.277594 12434 tablet_service.cc:1467] Processing CreateTablet for tablet bac0059e531d4156b96e978e5e610408 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240503 08:03:30.279667 12512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3958d27365e7439494639369ef57cd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.282034 12506 tablet_service.cc:1467] Processing CreateTablet for tablet 0968e24516804ba8aa9c02c264039dc2 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240503 08:03:30.284052 12432 tablet_service.cc:1467] Processing CreateTablet for tablet dacbaaaa60fd496fa2d2c195e8a2210c (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240503 08:03:30.286612 12509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac0059e531d4156b96e978e5e610408. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.288970 12436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce18a893675482c9720556a30c9ed10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.287950 12433 tablet_service.cc:1467] Processing CreateTablet for tablet d785e3daf54f4bb7b50647703534ecf0 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240503 08:03:30.287856 12507 tablet_service.cc:1467] Processing CreateTablet for tablet dacbaaaa60fd496fa2d2c195e8a2210c (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240503 08:03:30.294988 12431 tablet_service.cc:1467] Processing CreateTablet for tablet 0968e24516804ba8aa9c02c264039dc2 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240503 08:03:30.305258 12508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d785e3daf54f4bb7b50647703534ecf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.303997 12505 tablet_service.cc:1467] Processing CreateTablet for tablet 7e7df304e0a848f787eea813c111b4f3 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240503 08:03:30.307811 12434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac0059e531d4156b96e978e5e610408. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.298563 12430 tablet_service.cc:1467] Processing CreateTablet for tablet 7e7df304e0a848f787eea813c111b4f3 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240503 08:03:30.307953 12507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dacbaaaa60fd496fa2d2c195e8a2210c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.311038 12432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dacbaaaa60fd496fa2d2c195e8a2210c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.312731 12430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e7df304e0a848f787eea813c111b4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.318701 12506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0968e24516804ba8aa9c02c264039dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.324923 12505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e7df304e0a848f787eea813c111b4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.327689 12435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772658b6e692478fae18294150f1f32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.336444 12504 tablet_service.cc:1467] Processing CreateTablet for tablet 8474bb05d88f4292a5d2a597cab5042f (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240503 08:03:30.337937 12504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8474bb05d88f4292a5d2a597cab5042f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.334089 12429 tablet_service.cc:1467] Processing CreateTablet for tablet 8474bb05d88f4292a5d2a597cab5042f (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240503 08:03:30.340914 12431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0968e24516804ba8aa9c02c264039dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.344419 12429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8474bb05d88f4292a5d2a597cab5042f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.350417 12433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d785e3daf54f4bb7b50647703534ecf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.362429 12586 tablet_service.cc:1467] Processing CreateTablet for tablet 3958d27365e7439494639369ef57cd9b (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240503 08:03:30.368295 12586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3958d27365e7439494639369ef57cd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.373191 12585 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce18a893675482c9720556a30c9ed10 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240503 08:03:30.376754 12585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce18a893675482c9720556a30c9ed10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.385238 12584 tablet_service.cc:1467] Processing CreateTablet for tablet 772658b6e692478fae18294150f1f32a (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240503 08:03:30.405652 12583 tablet_service.cc:1467] Processing CreateTablet for tablet bac0059e531d4156b96e978e5e610408 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240503 08:03:30.412359 12582 tablet_service.cc:1467] Processing CreateTablet for tablet d785e3daf54f4bb7b50647703534ecf0 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240503 08:03:30.439882 12581 tablet_service.cc:1467] Processing CreateTablet for tablet dacbaaaa60fd496fa2d2c195e8a2210c (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240503 08:03:30.445835 12579 tablet_service.cc:1467] Processing CreateTablet for tablet 7e7df304e0a848f787eea813c111b4f3 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240503 08:03:30.446977 12584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772658b6e692478fae18294150f1f32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.447336 12580 tablet_service.cc:1467] Processing CreateTablet for tablet 0968e24516804ba8aa9c02c264039dc2 (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240503 08:03:30.450346 12580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0968e24516804ba8aa9c02c264039dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.452128 12578 tablet_service.cc:1467] Processing CreateTablet for tablet 8474bb05d88f4292a5d2a597cab5042f (DEFAULT_TABLE table=test-workload [id=7cdb53d84b5e4c9490cb7a9e0655d6be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240503 08:03:30.454522 12579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e7df304e0a848f787eea813c111b4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.456027 12583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac0059e531d4156b96e978e5e610408. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.457500 12582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d785e3daf54f4bb7b50647703534ecf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.459733 12578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8474bb05d88f4292a5d2a597cab5042f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.461256 12581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dacbaaaa60fd496fa2d2c195e8a2210c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:30.621456 12639 tablet_bootstrap.cc:492] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:30.628887 12472 heartbeater.cc:503] Master 127.12.6.62:36233 was elected leader, sending a full tablet report...
I20240503 08:03:30.652189 12640 tablet_bootstrap.cc:492] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:30.678498 12641 tablet_bootstrap.cc:492] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:30.687753 12640 tablet_bootstrap.cc:654] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.691027 12639 tablet_bootstrap.cc:654] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.728261 12641 tablet_bootstrap.cc:654] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.743639 12639 tablet_bootstrap.cc:492] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:30.744310 12639 ts_tablet_manager.cc:1367] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.144s user 0.010s sys 0.013s
I20240503 08:03:30.745358 12640 tablet_bootstrap.cc:492] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:30.743734 12641 tablet_bootstrap.cc:492] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:30.746531 12640 ts_tablet_manager.cc:1367] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.106s user 0.019s sys 0.007s
I20240503 08:03:30.746860 12641 ts_tablet_manager.cc:1367] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.069s user 0.021s sys 0.000s
I20240503 08:03:30.748225 12639 raft_consensus.cc:348] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.749398 12639 raft_consensus.cc:374] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.749723 12639 raft_consensus.cc:729] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.750172 12640 raft_consensus.cc:348] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.750928 12640 raft_consensus.cc:374] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.750583 12639 consensus_queue.cc:260] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.750705 12641 raft_consensus.cc:348] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.751344 12640 raft_consensus.cc:729] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.751776 12641 raft_consensus.cc:374] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.752213 12641 raft_consensus.cc:729] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.752203 12640 consensus_queue.cc:260] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.753540 12641 consensus_queue.cc:260] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.756304 12547 heartbeater.cc:503] Master 127.12.6.62:36233 was elected leader, sending a full tablet report...
I20240503 08:03:30.757198 12639 ts_tablet_manager.cc:1398] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.013s user 0.008s sys 0.003s
I20240503 08:03:30.758553 12639 tablet_bootstrap.cc:492] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:30.763083 12621 heartbeater.cc:503] Master 127.12.6.62:36233 was elected leader, sending a full tablet report...
I20240503 08:03:30.765089 12641 ts_tablet_manager.cc:1398] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.018s user 0.010s sys 0.004s
I20240503 08:03:30.766104 12641 tablet_bootstrap.cc:492] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:30.767450 12639 tablet_bootstrap.cc:654] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.771944 12640 ts_tablet_manager.cc:1398] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.025s user 0.013s sys 0.004s
I20240503 08:03:30.773166 12640 tablet_bootstrap.cc:492] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:30.775453 12641 tablet_bootstrap.cc:654] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.779716 12640 tablet_bootstrap.cc:654] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
W20240503 08:03:30.780694 12622 tablet.cc:2369] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 08:03:30.786404 12645 raft_consensus.cc:482] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:30.786916 12645 raft_consensus.cc:504] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.789906 12645 leader_election.cc:288] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:30.797380 12640 tablet_bootstrap.cc:492] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:30.797874 12640 ts_tablet_manager.cc:1367] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.008s
I20240503 08:03:30.803833 12640 raft_consensus.cc:348] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.804646 12640 raft_consensus.cc:374] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.804952 12640 raft_consensus.cc:729] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.806656 12640 consensus_queue.cc:260] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.809298 12640 ts_tablet_manager.cc:1398] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20240503 08:03:30.819619 12640 tablet_bootstrap.cc:492] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:30.828940 12640 tablet_bootstrap.cc:654] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.866345 12639 tablet_bootstrap.cc:492] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:30.866961 12639 ts_tablet_manager.cc:1367] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.109s user 0.006s sys 0.014s
I20240503 08:03:30.869980 12639 raft_consensus.cc:348] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.873198 12641 tablet_bootstrap.cc:492] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:30.873831 12641 ts_tablet_manager.cc:1367] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.108s user 0.027s sys 0.053s
I20240503 08:03:30.873267 12639 raft_consensus.cc:374] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.885955 12639 raft_consensus.cc:729] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.882009 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:30.888387 12639 consensus_queue.cc:260] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.886958 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:30.889606 12640 tablet_bootstrap.cc:492] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:30.890060 12640 ts_tablet_manager.cc:1367] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.071s user 0.017s sys 0.049s
I20240503 08:03:30.877441 12641 raft_consensus.cc:348] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
W20240503 08:03:30.891304 12485 leader_election.cc:341] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:30.891654 12639 ts_tablet_manager.cc:1398] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20240503 08:03:30.892091 12641 raft_consensus.cc:374] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.892500 12641 raft_consensus.cc:729] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.893258 12641 consensus_queue.cc:260] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.895777 12641 ts_tablet_manager.cc:1398] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.022s user 0.004s sys 0.003s
W20240503 08:03:30.896541 12485 leader_election.cc:341] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:30.897369 12485 leader_election.cc:302] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0; no voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:30.898242 12645 raft_consensus.cc:2726] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:30.899655 12639 tablet_bootstrap.cc:492] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:30.894600 12640 raft_consensus.cc:348] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.902585 12640 raft_consensus.cc:374] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.902935 12640 raft_consensus.cc:729] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.903693 12640 consensus_queue.cc:260] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.906050 12640 ts_tablet_manager.cc:1398] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.016s user 0.007s sys 0.001s
I20240503 08:03:30.907059 12640 tablet_bootstrap.cc:492] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:30.910331 12639 tablet_bootstrap.cc:654] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.913620 12640 tablet_bootstrap.cc:654] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.905972 12641 tablet_bootstrap.cc:492] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:30.925540 12639 tablet_bootstrap.cc:492] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:30.926189 12639 ts_tablet_manager.cc:1367] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.008s
I20240503 08:03:30.929167 12641 tablet_bootstrap.cc:654] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.929167 12639 raft_consensus.cc:348] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.930318 12639 raft_consensus.cc:374] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.930717 12639 raft_consensus.cc:729] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.931551 12639 consensus_queue.cc:260] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.932173 12646 raft_consensus.cc:482] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:30.932710 12646 raft_consensus.cc:504] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.934392 12639 ts_tablet_manager.cc:1398] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20240503 08:03:30.935439 12639 tablet_bootstrap.cc:492] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:30.937561 12646 leader_election.cc:288] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:30.947046 12640 tablet_bootstrap.cc:492] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:30.947834 12640 ts_tablet_manager.cc:1367] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.009s
I20240503 08:03:30.952430 12646 raft_consensus.cc:482] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:30.952951 12646 raft_consensus.cc:504] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.958072 12646 leader_election.cc:288] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:30.961445 12639 tablet_bootstrap.cc:654] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:30.958842 12640 raft_consensus.cc:348] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.963270 12640 raft_consensus.cc:374] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.963645 12640 raft_consensus.cc:729] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.964902 12640 consensus_queue.cc:260] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.975764 12640 ts_tablet_manager.cc:1398] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.028s user 0.007s sys 0.005s
I20240503 08:03:30.979982 12640 tablet_bootstrap.cc:492] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:30.985980 12641 tablet_bootstrap.cc:492] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:30.986517 12641 ts_tablet_manager.cc:1367] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.081s user 0.015s sys 0.007s
I20240503 08:03:30.991359 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:30.993221 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:30.993016 12641 raft_consensus.cc:348] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:30.994164 12641 raft_consensus.cc:374] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:30.994323 12521 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 0.
I20240503 08:03:30.994560 12641 raft_consensus.cc:729] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:30.998518 12641 consensus_queue.cc:260] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
W20240503 08:03:31.000044 12412 leader_election.cc:341] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.000612 12640 tablet_bootstrap.cc:654] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.002734 12412 leader_election.cc:302] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.002622 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.001866 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.004539 12647 raft_consensus.cc:482] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.004703 12646 raft_consensus.cc:2781] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.005200 12646 raft_consensus.cc:482] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.005368 12595 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 0.
I20240503 08:03:31.005604 12646 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:03:31.006668 12410 leader_election.cc:341] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.008376 12410 leader_election.cc:302] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13; no voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:31.009985 12639 tablet_bootstrap.cc:492] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:31.010596 12639 ts_tablet_manager.cc:1367] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.075s user 0.013s sys 0.023s
I20240503 08:03:31.005198 12647 raft_consensus.cc:504] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.017484 12641 ts_tablet_manager.cc:1398] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.031s user 0.002s sys 0.009s
I20240503 08:03:31.017501 12647 leader_election.cc:288] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.024435 12641 tablet_bootstrap.cc:492] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:31.034250 12662 raft_consensus.cc:2726] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:31.036202 12639 raft_consensus.cc:348] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.037204 12639 raft_consensus.cc:374] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.037573 12639 raft_consensus.cc:729] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.039602 12639 consensus_queue.cc:260] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.041029 12646 raft_consensus.cc:504] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.044397 12647 raft_consensus.cc:482] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.044843 12647 raft_consensus.cc:504] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.046267 12639 ts_tablet_manager.cc:1398] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.035s user 0.016s sys 0.000s
I20240503 08:03:31.046873 12647 leader_election.cc:288] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.049280 12639 tablet_bootstrap.cc:492] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:31.050848 12646 leader_election.cc:288] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.052110 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:31.053067 12595 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.052978 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:31.055655 12522 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.065523 12522 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 1.
I20240503 08:03:31.066828 12412 leader_election.cc:302] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.067822 12646 raft_consensus.cc:2781] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.068485 12646 raft_consensus.cc:686] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:31.069579 12646 consensus_queue.cc:237] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.085848 12595 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 1.
I20240503 08:03:31.088403 12641 tablet_bootstrap.cc:654] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.092847 12640 tablet_bootstrap.cc:492] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:31.093904 12640 ts_tablet_manager.cc:1367] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.114s user 0.028s sys 0.018s
I20240503 08:03:31.097043 12640 raft_consensus.cc:348] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.097841 12640 raft_consensus.cc:374] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.098129 12640 raft_consensus.cc:729] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.098873 12640 consensus_queue.cc:260] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.101291 12640 ts_tablet_manager.cc:1398] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240503 08:03:31.102291 12640 tablet_bootstrap.cc:492] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:31.106372 12639 tablet_bootstrap.cc:654] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.108754 12646 raft_consensus.cc:482] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.109808 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:31.111864 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
W20240503 08:03:31.112999 12561 leader_election.cc:341] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:03:31.116312 12561 leader_election.cc:341] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.110091 12646 raft_consensus.cc:504] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.119925 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
W20240503 08:03:31.121897 12559 leader_election.cc:341] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.122548 12559 leader_election.cc:302] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d318d4aca45244ba9cd5a8cb36cab286; no voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:31.124028 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.124855 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.125162 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
W20240503 08:03:31.126248 12410 leader_election.cc:341] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:03:31.128425 12412 leader_election.cc:341] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.128500 12647 raft_consensus.cc:2726] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:31.126299 12646 leader_election.cc:288] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.128940 12412 leader_election.cc:302] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13; no voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286
W20240503 08:03:31.130599 12559 leader_election.cc:341] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.131208 12646 raft_consensus.cc:2726] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:31.132730 12559 leader_election.cc:302] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d318d4aca45244ba9cd5a8cb36cab286; no voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:31.126890 12343 catalog_manager.cc:5484] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:31.134171 12647 raft_consensus.cc:2726] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:31.134348 12640 tablet_bootstrap.cc:654] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.136734 12639 tablet_bootstrap.cc:492] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:31.136929 12641 tablet_bootstrap.cc:492] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:31.137225 12639 ts_tablet_manager.cc:1367] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.090s user 0.013s sys 0.006s
I20240503 08:03:31.137354 12641 ts_tablet_manager.cc:1367] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.113s user 0.019s sys 0.027s
I20240503 08:03:31.142762 12639 raft_consensus.cc:348] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.143765 12639 raft_consensus.cc:374] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.144063 12639 raft_consensus.cc:729] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.144855 12639 consensus_queue.cc:260] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.149163 12639 ts_tablet_manager.cc:1398] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20240503 08:03:31.144157 12641 raft_consensus.cc:348] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.150573 12641 raft_consensus.cc:374] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.151157 12641 raft_consensus.cc:729] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.151938 12641 consensus_queue.cc:260] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.152583 12639 tablet_bootstrap.cc:492] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:31.154289 12641 ts_tablet_manager.cc:1398] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20240503 08:03:31.157191 12641 tablet_bootstrap.cc:492] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:31.162034 12640 tablet_bootstrap.cc:492] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:31.162586 12640 ts_tablet_manager.cc:1367] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.060s user 0.011s sys 0.005s
I20240503 08:03:31.164461 12641 tablet_bootstrap.cc:654] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.166211 12640 raft_consensus.cc:348] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.166993 12640 raft_consensus.cc:374] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.167768 12640 raft_consensus.cc:729] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.168526 12640 consensus_queue.cc:260] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.176414 12640 ts_tablet_manager.cc:1398] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.013s user 0.004s sys 0.005s
I20240503 08:03:31.177770 12640 tablet_bootstrap.cc:492] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:31.170317 12639 tablet_bootstrap.cc:654] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.183110 12641 tablet_bootstrap.cc:492] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:31.183763 12641 ts_tablet_manager.cc:1367] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.027s user 0.015s sys 0.005s
I20240503 08:03:31.188745 12641 raft_consensus.cc:348] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.189734 12641 raft_consensus.cc:374] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.190075 12641 raft_consensus.cc:729] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.190462 12640 tablet_bootstrap.cc:654] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.191004 12641 consensus_queue.cc:260] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.194181 12641 ts_tablet_manager.cc:1398] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20240503 08:03:31.195447 12641 tablet_bootstrap.cc:492] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:31.197219 12639 tablet_bootstrap.cc:492] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:31.197711 12639 ts_tablet_manager.cc:1367] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.045s user 0.012s sys 0.002s
I20240503 08:03:31.200443 12639 raft_consensus.cc:348] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.201192 12639 raft_consensus.cc:374] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.201478 12639 raft_consensus.cc:729] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.202168 12641 tablet_bootstrap.cc:654] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.202241 12639 consensus_queue.cc:260] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.205121 12645 raft_consensus.cc:482] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.205571 12645 raft_consensus.cc:504] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.206411 12639 ts_tablet_manager.cc:1398] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20240503 08:03:31.207552 12645 leader_election.cc:288] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.208684 12639 tablet_bootstrap.cc:492] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:31.208648 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.208468 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.209653 12446 raft_consensus.cc:2454] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
W20240503 08:03:31.210430 12485 leader_election.cc:341] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.211186 12485 leader_election.cc:302] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters: d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:31.214876 12645 raft_consensus.cc:2781] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.215364 12645 raft_consensus.cc:482] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.215788 12645 raft_consensus.cc:3037] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.224601 12641 tablet_bootstrap.cc:492] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:31.225282 12641 ts_tablet_manager.cc:1367] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.015s
I20240503 08:03:31.226836 12646 raft_consensus.cc:482] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.227367 12646 raft_consensus.cc:504] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.229409 12645 raft_consensus.cc:504] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.230628 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.231297 12645 leader_election.cc:288] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
W20240503 08:03:31.231369 12392 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20240503 08:03:31.231894 12410 leader_election.cc:341] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.229406 12646 leader_election.cc:288] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.232647 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:31.232705 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:31.233134 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.233693 12446 raft_consensus.cc:3037] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.229130 12641 raft_consensus.cc:348] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.234344 12641 raft_consensus.cc:374] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 08:03:31.234305 12412 leader_election.cc:341] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.235369 12641 raft_consensus.cc:729] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.235915 12412 leader_election.cc:302] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13; no voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:31.236909 12641 consensus_queue.cc:260] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.237767 12640 tablet_bootstrap.cc:492] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:31.237947 12595 raft_consensus.cc:3037] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.238237 12640 ts_tablet_manager.cc:1367] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.061s user 0.008s sys 0.016s
I20240503 08:03:31.238654 12646 raft_consensus.cc:2726] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:31.238895 12645 raft_consensus.cc:482] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.239423 12645 raft_consensus.cc:504] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.244446 12640 raft_consensus.cc:348] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.245477 12640 raft_consensus.cc:374] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.245555 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.246007 12640 raft_consensus.cc:729] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.246307 12639 tablet_bootstrap.cc:654] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.246344 12447 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.246701 12640 consensus_queue.cc:260] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.247589 12645 leader_election.cc:288] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.248597 12485 leader_election.cc:302] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.249380 12645 raft_consensus.cc:2781] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.249732 12645 raft_consensus.cc:482] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.250046 12645 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.249900 12446 raft_consensus.cc:2454] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.250196 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.255540 12596 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.253264 12595 raft_consensus.cc:2454] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.257169 12645 raft_consensus.cc:504] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.259702 12645 leader_election.cc:288] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.260512 12485 leader_election.cc:302] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.261044 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.261704 12447 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.264709 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:31.265340 12596 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.263046 12641 ts_tablet_manager.cc:1398] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.037s user 0.009s sys 0.001s
I20240503 08:03:31.269112 12447 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.269846 12641 tablet_bootstrap.cc:492] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:31.270120 12645 raft_consensus.cc:2781] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.273939 12645 raft_consensus.cc:686] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:31.274628 12640 ts_tablet_manager.cc:1398] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.036s user 0.003s sys 0.003s
I20240503 08:03:31.274860 12645 consensus_queue.cc:237] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.281067 12640 tablet_bootstrap.cc:492] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:31.278457 12485 leader_election.cc:302] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.290542 12596 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.293360 12639 tablet_bootstrap.cc:492] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:31.293882 12639 ts_tablet_manager.cc:1367] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.085s user 0.013s sys 0.004s
I20240503 08:03:31.294906 12344 catalog_manager.cc:5484] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:31.298017 12639 raft_consensus.cc:348] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.298884 12639 raft_consensus.cc:374] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.299636 12639 raft_consensus.cc:729] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.300388 12640 tablet_bootstrap.cc:654] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.301792 12641 tablet_bootstrap.cc:654] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.302290 12645 raft_consensus.cc:2781] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.300922 12639 consensus_queue.cc:260] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.304636 12645 raft_consensus.cc:686] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:31.306775 12639 ts_tablet_manager.cc:1398] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.013s user 0.002s sys 0.006s
I20240503 08:03:31.308320 12639 tablet_bootstrap.cc:492] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:31.305505 12645 consensus_queue.cc:237] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.313143 12640 tablet_bootstrap.cc:492] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:31.313848 12640 ts_tablet_manager.cc:1367] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.006s
I20240503 08:03:31.317279 12639 tablet_bootstrap.cc:654] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.317243 12640 raft_consensus.cc:348] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.318331 12640 raft_consensus.cc:374] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.318701 12640 raft_consensus.cc:729] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.319813 12640 consensus_queue.cc:260] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.322463 12641 tablet_bootstrap.cc:492] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:31.322957 12641 ts_tablet_manager.cc:1367] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.053s user 0.009s sys 0.004s
I20240503 08:03:31.325662 12641 raft_consensus.cc:348] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.326432 12641 raft_consensus.cc:374] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.326714 12641 raft_consensus.cc:729] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.327555 12641 consensus_queue.cc:260] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.330035 12641 ts_tablet_manager.cc:1398] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20240503 08:03:31.332731 12641 tablet_bootstrap.cc:492] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:31.332742 12639 tablet_bootstrap.cc:492] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:31.333452 12639 ts_tablet_manager.cc:1367] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.011s
I20240503 08:03:31.336441 12639 raft_consensus.cc:348] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.337427 12639 raft_consensus.cc:374] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.338038 12639 raft_consensus.cc:729] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.341981 12640 ts_tablet_manager.cc:1398] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20240503 08:03:31.344278 12640 tablet_bootstrap.cc:492] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13: Bootstrap starting.
I20240503 08:03:31.348851 12641 tablet_bootstrap.cc:654] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.351449 12639 consensus_queue.cc:260] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.350958 12344 catalog_manager.cc:5484] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:31.353950 12639 ts_tablet_manager.cc:1398] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20240503 08:03:31.354832 12639 tablet_bootstrap.cc:492] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: Bootstrap starting.
I20240503 08:03:31.361639 12639 tablet_bootstrap.cc:654] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.363883 12640 tablet_bootstrap.cc:654] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.377691 12639 tablet_bootstrap.cc:492] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: No bootstrap required, opened a new log
I20240503 08:03:31.378201 12639 ts_tablet_manager.cc:1367] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.011s
I20240503 08:03:31.380715 12639 raft_consensus.cc:348] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.381469 12639 raft_consensus.cc:374] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.381753 12639 raft_consensus.cc:729] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.382377 12639 consensus_queue.cc:260] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.382987 12641 tablet_bootstrap.cc:492] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:31.384040 12641 ts_tablet_manager.cc:1367] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.010s
I20240503 08:03:31.386447 12641 raft_consensus.cc:348] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.387621 12641 raft_consensus.cc:374] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.388370 12639 ts_tablet_manager.cc:1398] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20240503 08:03:31.387883 12641 raft_consensus.cc:729] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.389477 12641 consensus_queue.cc:260] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.391534 12641 ts_tablet_manager.cc:1398] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240503 08:03:31.392707 12641 tablet_bootstrap.cc:492] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286: Bootstrap starting.
I20240503 08:03:31.398558 12641 tablet_bootstrap.cc:654] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:31.398667 12645 raft_consensus.cc:482] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.399504 12640 tablet_bootstrap.cc:492] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13: No bootstrap required, opened a new log
I20240503 08:03:31.399335 12645 raft_consensus.cc:504] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.399921 12640 ts_tablet_manager.cc:1367] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13: Time spent bootstrapping tablet: real 0.056s user 0.014s sys 0.002s
I20240503 08:03:31.401377 12645 leader_election.cc:288] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.403079 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.403247 12640 raft_consensus.cc:348] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.404001 12640 raft_consensus.cc:374] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.403085 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.404291 12640 raft_consensus.cc:729] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Initialized, Role: FOLLOWER
W20240503 08:03:31.405308 12485 leader_election.cc:341] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.407500 12640 consensus_queue.cc:260] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.410650 12446 raft_consensus.cc:2454] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.413148 12485 leader_election.cc:302] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters: d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:31.415035 12645 raft_consensus.cc:2781] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.415789 12645 raft_consensus.cc:482] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.416183 12645 raft_consensus.cc:3037] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.416278 12647 raft_consensus.cc:482] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.416766 12647 raft_consensus.cc:504] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.421214 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.421948 12446 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 0.
I20240503 08:03:31.422897 12640 ts_tablet_manager.cc:1398] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20240503 08:03:31.423252 12559 leader_election.cc:302] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.424682 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:31.418988 12647 leader_election.cc:288] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.428380 12647 raft_consensus.cc:2781] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.428874 12647 raft_consensus.cc:482] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.429271 12647 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.426977 12678 raft_consensus.cc:482] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.431614 12678 raft_consensus.cc:504] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.434517 12678 leader_election.cc:288] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.426653 12645 raft_consensus.cc:504] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.428347 12521 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 0.
I20240503 08:03:31.437902 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.438210 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.438609 12596 raft_consensus.cc:2454] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.438751 12447 raft_consensus.cc:3037] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.440984 12645 leader_election.cc:288] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.444180 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.445101 12446 raft_consensus.cc:2454] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.446074 12641 tablet_bootstrap.cc:492] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286: No bootstrap required, opened a new log
I20240503 08:03:31.446563 12641 ts_tablet_manager.cc:1367] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286: Time spent bootstrapping tablet: real 0.054s user 0.032s sys 0.018s
I20240503 08:03:31.447907 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:31.448004 12485 leader_election.cc:302] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.449366 12641 raft_consensus.cc:348] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.449739 12645 raft_consensus.cc:2781] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.450080 12641 raft_consensus.cc:374] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:31.450132 12645 raft_consensus.cc:482] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.450469 12641 raft_consensus.cc:729] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Initialized, Role: FOLLOWER
I20240503 08:03:31.450654 12645 raft_consensus.cc:3037] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:03:31.451306 12485 leader_election.cc:341] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:03:31.456902 12678 raft_consensus.cc:482] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.457245 12645 raft_consensus.cc:504] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.457504 12678 raft_consensus.cc:504] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.451175 12641 consensus_queue.cc:260] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.463719 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.464210 12641 ts_tablet_manager.cc:1398] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286: Time spent starting tablet: real 0.017s user 0.004s sys 0.004s
I20240503 08:03:31.466081 12447 raft_consensus.cc:2454] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.464560 12445 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.465343 12647 raft_consensus.cc:504] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.465104 12678 leader_election.cc:288] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.471859 12647 leader_election.cc:288] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.472637 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.472641 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:31.473357 12445 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.473459 12596 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 0.
I20240503 08:03:31.476606 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.479398 12446 raft_consensus.cc:3037] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.478870 12645 leader_election.cc:288] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.480341 12485 leader_election.cc:302] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.480571 12688 raft_consensus.cc:482] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.485157 12688 raft_consensus.cc:504] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.486865 12688 leader_election.cc:288] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613)
I20240503 08:03:31.484437 12445 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 1.
I20240503 08:03:31.487877 12446 raft_consensus.cc:2454] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.488461 12559 leader_election.cc:302] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.489300 12688 raft_consensus.cc:2781] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.489264 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:31.489570 12647 raft_consensus.cc:482] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.490026 12521 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.490080 12647 raft_consensus.cc:504] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.490615 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:31.491302 12522 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 0.
I20240503 08:03:31.492472 12485 leader_election.cc:302] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters: d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:31.492372 12647 leader_election.cc:288] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.493534 12678 raft_consensus.cc:2781] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.493898 12678 raft_consensus.cc:482] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.494244 12678 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.494993 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.495731 12446 raft_consensus.cc:2454] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 0.
I20240503 08:03:31.496918 12559 leader_election.cc:302] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.497291 12521 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 1.
I20240503 08:03:31.497844 12647 raft_consensus.cc:2781] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.498296 12647 raft_consensus.cc:482] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.498710 12647 raft_consensus.cc:3037] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.503361 12645 raft_consensus.cc:2781] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.506546 12647 raft_consensus.cc:504] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.505678 12678 raft_consensus.cc:504] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.509248 12678 leader_election.cc:288] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:31.509800 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.507375 12485 leader_election.cc:302] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:31.509904 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.510556 12446 raft_consensus.cc:3037] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.511005 12447 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 0.
I20240503 08:03:31.512414 12559 leader_election.cc:302] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.515116 12678 raft_consensus.cc:2781] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.515679 12678 raft_consensus.cc:686] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:31.524376 12647 leader_election.cc:288] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.525182 12647 raft_consensus.cc:2781] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:03:31.525564 12647 raft_consensus.cc:482] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:03:31.525655 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:31.525967 12647 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.526289 12596 raft_consensus.cc:3037] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.529593 12446 raft_consensus.cc:2454] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 1.
I20240503 08:03:31.532375 12688 raft_consensus.cc:686] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:31.533356 12688 consensus_queue.cc:237] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.534296 12596 raft_consensus.cc:2454] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.545785 12559 leader_election.cc:302] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.547504 12689 raft_consensus.cc:2781] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.548916 12689 raft_consensus.cc:686] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:31.551373 12689 consensus_queue.cc:237] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.549860 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.553951 12446 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.553830 12647 raft_consensus.cc:504] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.520094 12678 consensus_queue.cc:237] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.558358 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:31.560566 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:31.560731 12522 raft_consensus.cc:2454] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 0.
I20240503 08:03:31.556042 12645 raft_consensus.cc:686] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:31.561244 12520 raft_consensus.cc:3037] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.561878 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:31.562007 12645 consensus_queue.cc:237] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.562831 12596 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.571295 12646 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.575778 12647 leader_election.cc:288] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613)
I20240503 08:03:31.576777 12520 raft_consensus.cc:2454] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 1.
I20240503 08:03:31.582051 12447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:31.582078 12596 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.583480 12447 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.584339 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:31.584616 12646 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.584976 12520 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.585155 12485 leader_election.cc:302] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.588771 12344 catalog_manager.cc:5484] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:31.593562 12645 raft_consensus.cc:2781] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.595064 12645 raft_consensus.cc:686] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:31.596539 12645 consensus_queue.cc:237] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.598598 12520 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 1.
I20240503 08:03:31.602738 12561 leader_election.cc:302] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:31.611510 12647 raft_consensus.cc:2781] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:31.623276 12647 raft_consensus.cc:686] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:31.624400 12647 consensus_queue.cc:237] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.634117 12343 catalog_manager.cc:5484] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 0 to 1, leader changed from <none> to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:31.642484 12446 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 1.
I20240503 08:03:31.629020 12447 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 1.
I20240503 08:03:31.671828 12343 catalog_manager.cc:5484] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:31.676918 12343 catalog_manager.cc:5484] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:31.694371 12344 catalog_manager.cc:5484] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 0 to 1, leader changed from <none> to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:31.698580 12344 catalog_manager.cc:5484] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 0 to 1, leader changed from <none> to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:31.699815 12645 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.703847 12645 consensus_queue.cc:1035] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.717806 12690 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.745360 12691 consensus_queue.cc:1035] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:03:31.899708 12647 raft_consensus.cc:482] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:03:31.900169 12647 raft_consensus.cc:504] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:31.901772 12647 leader_election.cc:288] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:31.902832 12446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:31.902932 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:31.903882 12446 raft_consensus.cc:2379] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d318d4aca45244ba9cd5a8cb36cab286 in current term 1: Already voted for candidate 6bbad1351d304f2da4f413e6522c0bd0 in this term.
I20240503 08:03:31.904907 12559 leader_election.cc:302] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d318d4aca45244ba9cd5a8cb36cab286; no voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:31.905612 12647 raft_consensus.cc:3037] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:31.910288 12647 raft_consensus.cc:2726] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:03:31.951568 12647 consensus_queue.cc:1035] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.963310 12692 consensus_queue.cc:1035] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:31.999403 12678 consensus_queue.cc:1035] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.007973 12595 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3958d27365e7439494639369ef57cd9b"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
mode: GRACEFUL
new_leader_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:55388
I20240503 08:03:32.008729 12595 raft_consensus.cc:595] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Received request to transfer leadership to TS 49e3300ff51e489889daf9f659d10b13
I20240503 08:03:32.015338 12678 consensus_queue.cc:1035] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.038908 12521 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:44206
I20240503 08:03:32.039515 12521 raft_consensus.cc:595] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Received request to transfer leadership to TS 49e3300ff51e489889daf9f659d10b13
I20240503 08:03:32.045344 12690 consensus_queue.cc:1035] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.049335 12596 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bac0059e531d4156b96e978e5e610408"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
mode: GRACEFUL
new_leader_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:55388
I20240503 08:03:32.049997 12596 raft_consensus.cc:595] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Received request to transfer leadership to TS 49e3300ff51e489889daf9f659d10b13
I20240503 08:03:32.058488 12692 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.062971 12690 consensus_queue.cc:1035] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.069741 12692 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.071658 12688 raft_consensus.cc:979] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: : Instructing follower 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:32.072808 12647 raft_consensus.cc:1067] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Signalling peer 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:32.075078 12445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bac0059e531d4156b96e978e5e610408"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:45784
I20240503 08:03:32.083928 12445 raft_consensus.cc:482] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:32.084371 12445 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.092909 12692 consensus_queue.cc:1035] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.107453 12445 raft_consensus.cc:504] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.116222 12691 raft_consensus.cc:979] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:32.116688 12690 raft_consensus.cc:1067] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Signalling peer 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:32.118412 12446 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:45768
I20240503 08:03:32.119066 12446 raft_consensus.cc:482] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:32.119495 12446 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.125648 12692 consensus_queue.cc:1035] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.126152 12446 raft_consensus.cc:504] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.126942 12691 consensus_queue.cc:1035] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.130362 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:32.130964 12520 raft_consensus.cc:3032] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:32.131314 12520 raft_consensus.cc:729] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:32.132092 12520 consensus_queue.cc:260] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.133383 12520 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.140657 12520 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:32.141845 12412 leader_election.cc:302] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:32.143368 12704 raft_consensus.cc:2781] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:32.143875 12704 raft_consensus.cc:686] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:32.145294 12704 consensus_queue.cc:237] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.151305 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:32.153730 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:32.154346 12595 raft_consensus.cc:3032] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:32.154680 12595 raft_consensus.cc:729] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:32.155190 12445 leader_election.cc:288] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:32.155489 12595 consensus_queue.cc:260] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.157415 12688 raft_consensus.cc:979] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: : Instructing follower 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:32.157529 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:32.161556 12595 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.161702 12596 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.165999 12688 raft_consensus.cc:1067] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Signalling peer 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:32.167861 12443 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3958d27365e7439494639369ef57cd9b"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:45784
I20240503 08:03:32.167707 12691 consensus_queue.cc:1035] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.168339 12596 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:32.169593 12443 raft_consensus.cc:482] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:32.170019 12443 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.170605 12595 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:32.173177 12344 catalog_manager.cc:5484] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 1 to 2, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:32.158849 12446 leader_election.cc:288] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:32.181442 12443 raft_consensus.cc:504] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.186187 12443 leader_election.cc:288] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:32.186800 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:32.187891 12520 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.190330 12410 leader_election.cc:302] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters: 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:32.190780 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:32.191591 12595 raft_consensus.cc:3032] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:32.191848 12704 raft_consensus.cc:2781] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:32.191957 12595 raft_consensus.cc:729] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:32.192665 12704 raft_consensus.cc:686] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:32.193204 12595 consensus_queue.cc:260] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.194357 12704 consensus_queue.cc:237] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.197155 12595 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.201076 12520 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:32.204667 12595 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:32.207175 12410 leader_election.cc:302] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:32.208388 12646 raft_consensus.cc:2781] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:32.209863 12646 raft_consensus.cc:686] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:32.210681 12646 consensus_queue.cc:237] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:32.212553 12344 catalog_manager.cc:5484] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 1 to 2, leader changed from d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3) to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:32.232614 12341 catalog_manager.cc:5484] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 1 to 2, leader changed from d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3) to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:32.722635 12521 raft_consensus.cc:1261] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:32.723964 12646 consensus_queue.cc:1035] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:32.730031 12596 raft_consensus.cc:1261] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:32.732748 12727 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:03:32.735335 12596 raft_consensus.cc:1261] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:32.742412 12727 consensus_queue.cc:1035] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20240503 08:03:32.771930 12521 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:32.776417 12520 raft_consensus.cc:1261] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:32.777770 12730 consensus_queue.cc:1035] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:32.780364 12521 raft_consensus.cc:1261] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:32.782227 12730 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:03:32.786861 12594 raft_consensus.cc:1261] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:32.788771 12731 consensus_queue.cc:1035] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:35.121356 12312 auto_leader_rebalancer-test.cc:261] The leader distribution is
I20240503 08:03:35.122153 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:35.138583 12445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "772658b6e692478fae18294150f1f32a"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
mode: GRACEFUL
new_leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:37030
I20240503 08:03:35.139166 12445 raft_consensus.cc:595] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 LEADER]: Received request to transfer leadership to TS d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:35.149331 12521 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:42360
I20240503 08:03:35.149859 12521 raft_consensus.cc:595] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Received request to transfer leadership to TS d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:35.150903 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240503 08:03:35.151351 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:35.254595 12740 raft_consensus.cc:979] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: : Instructing follower d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:35.255020 12740 raft_consensus.cc:1067] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 LEADER]: Signalling peer d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:35.256685 12594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "772658b6e692478fae18294150f1f32a"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:55386
I20240503 08:03:35.257133 12594 raft_consensus.cc:482] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:35.257385 12594 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:35.262005 12594 raft_consensus.cc:504] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:35.263526 12594 leader_election.cc:288] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 2 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:35.264472 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:35.264529 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:35.265147 12445 raft_consensus.cc:3032] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:35.265215 12521 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:35.265448 12445 raft_consensus.cc:729] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:35.266077 12445 consensus_queue.cc:260] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:35.266973 12445 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:35.269982 12521 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 2.
I20240503 08:03:35.271059 12561 leader_election.cc:302] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:35.271435 12445 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 2.
I20240503 08:03:35.271776 12734 raft_consensus.cc:2781] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:35.273121 12734 raft_consensus.cc:686] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:35.273969 12734 consensus_queue.cc:237] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:35.283737 12343 catalog_manager.cc:5484] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 1 to 2, leader changed from 49e3300ff51e489889daf9f659d10b13 (127.12.6.1) to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 2 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:35.527217 12691 raft_consensus.cc:979] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:35.527611 12691 raft_consensus.cc:1067] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Signalling peer d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:35.528906 12594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:55374
I20240503 08:03:35.529403 12594 raft_consensus.cc:482] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:35.529706 12594 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:35.543627 12594 raft_consensus.cc:504] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:35.545569 12594 leader_election.cc:288] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 2 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:35.546483 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:35.546536 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:35.547174 12445 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:35.547297 12521 raft_consensus.cc:3032] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:35.547643 12521 raft_consensus.cc:729] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:35.548278 12521 consensus_queue.cc:260] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:35.549330 12521 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:35.561549 12445 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 2.
I20240503 08:03:35.561554 12521 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 2.
I20240503 08:03:35.563257 12561 leader_election.cc:302] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:35.564074 12734 raft_consensus.cc:2781] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:35.564467 12734 raft_consensus.cc:686] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:35.565189 12734 consensus_queue.cc:237] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:35.572398 12343 catalog_manager.cc:5484] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 1 to 2, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 2 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:35.667965 12521 raft_consensus.cc:1261] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:35.669200 12734 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:35.678813 12445 raft_consensus.cc:1261] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:35.680667 12741 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:36.076483 12445 raft_consensus.cc:1261] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:36.077814 12741 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:36.080916 12521 raft_consensus.cc:1261] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:36.082664 12734 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:03:36.152156 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:36.157629 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:36.158058 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:37.158922 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:37.164534 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:37.166497 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:38.167718 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:38.173650 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:38.174230 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:39.175108 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:39.180373 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:39.180816 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:40.181654 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:40.186556 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:40.187002 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:41.187812 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:41.193426 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:41.193866 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:42.194741 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:42.199455 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:42.199899 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:43.200726 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:43.206853 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:43.207288 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:44.208132 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:44.212749 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:44.213184 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:45.236291 12312 auto_leader_rebalancer-test.cc:280] The leader distribution is
I20240503 08:03:45.250859 12445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3958d27365e7439494639369ef57cd9b"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
mode: GRACEFUL
new_leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:35796
I20240503 08:03:45.251631 12445 raft_consensus.cc:595] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Received request to transfer leadership to TS 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:45.256338 12445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
mode: GRACEFUL
new_leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:35796
I20240503 08:03:45.256852 12445 raft_consensus.cc:595] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Received request to transfer leadership to TS 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:45.261953 12596 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "772658b6e692478fae18294150f1f32a"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
mode: GRACEFUL
new_leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:55388
I20240503 08:03:45.262442 12596 raft_consensus.cc:595] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Received request to transfer leadership to TS 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:45.267158 12445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bac0059e531d4156b96e978e5e610408"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
mode: GRACEFUL
new_leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:35796
I20240503 08:03:45.267632 12445 raft_consensus.cc:595] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Received request to transfer leadership to TS 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:45.279570 12596 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
mode: GRACEFUL
new_leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:55388
I20240503 08:03:45.280088 12596 raft_consensus.cc:595] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Received request to transfer leadership to TS 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:45.346184 12740 raft_consensus.cc:979] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: : Instructing follower 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.346809 12767 raft_consensus.cc:1067] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Signalling peer 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.348503 12520 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3958d27365e7439494639369ef57cd9b"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:44192
I20240503 08:03:45.348961 12520 raft_consensus.cc:482] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:45.349284 12520 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.355177 12520 raft_consensus.cc:504] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.357378 12520 leader_election.cc:288] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:45.358095 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:45.358326 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3958d27365e7439494639369ef57cd9b" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:45.358765 12445 raft_consensus.cc:3032] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:03:45.358991 12596 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.359161 12445 raft_consensus.cc:729] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:45.360108 12445 consensus_queue.cc:260] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.361930 12445 raft_consensus.cc:3037] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.366026 12768 raft_consensus.cc:979] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: : Instructing follower 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.366559 12768 raft_consensus.cc:1067] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Signalling peer 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.368655 12521 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "772658b6e692478fae18294150f1f32a"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:44196
I20240503 08:03:45.369354 12521 raft_consensus.cc:482] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:45.369758 12521 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.369990 12596 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.370201 12445 raft_consensus.cc:2454] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.371745 12485 leader_election.cc:302] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:45.372582 12763 raft_consensus.cc:979] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: : Instructing follower 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.372798 12762 raft_consensus.cc:2781] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:03:45.374168 12762 raft_consensus.cc:686] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:45.373211 12763 raft_consensus.cc:1067] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Signalling peer 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.375785 12767 raft_consensus.cc:979] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: : Instructing follower 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.375237 12762 consensus_queue.cc:237] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.376209 12767 raft_consensus.cc:1067] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Signalling peer 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.376878 12521 raft_consensus.cc:504] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.377207 12520 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:44196
I20240503 08:03:45.377892 12520 raft_consensus.cc:482] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:45.378007 12522 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bac0059e531d4156b96e978e5e610408"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:44192
I20240503 08:03:45.378393 12520 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.378628 12522 raft_consensus.cc:482] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:45.379000 12522 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.385454 12520 raft_consensus.cc:504] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.385711 12522 raft_consensus.cc:504] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.385862 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:45.386390 12521 leader_election.cc:288] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:45.386626 12596 raft_consensus.cc:3032] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:03:45.387073 12596 raft_consensus.cc:729] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:45.387915 12520 leader_election.cc:288] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:45.388913 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:45.388639 12596 consensus_queue.cc:260] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.389402 12594 raft_consensus.cc:3032] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:03:45.389719 12594 raft_consensus.cc:729] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:45.390218 12522 leader_election.cc:288] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:45.390080 12596 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.391513 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:45.390551 12594 consensus_queue.cc:260] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.392156 12595 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.392112 12444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:45.392539 12443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:45.392822 12444 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.393170 12443 raft_consensus.cc:3032] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:03:45.393499 12443 raft_consensus.cc:729] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:45.394541 12443 consensus_queue.cc:260] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.398283 12443 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.399626 12341 catalog_manager.cc:5484] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 2 to 3, leader changed from 49e3300ff51e489889daf9f659d10b13 (127.12.6.1) to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 3 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:45.402755 12444 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.403227 12594 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.403928 12595 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.405203 12485 leader_election.cc:302] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0; no voters:
I20240503 08:03:45.406176 12762 raft_consensus.cc:2781] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:03:45.406082 12485 leader_election.cc:302] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:45.407371 12770 raft_consensus.cc:2781] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:03:45.408777 12770 raft_consensus.cc:686] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:45.409850 12594 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.409713 12770 consensus_queue.cc:237] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.411921 12596 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.410053 12443 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.413343 12485 leader_election.cc:302] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:45.406926 12762 raft_consensus.cc:686] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:45.414306 12773 raft_consensus.cc:2781] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:03:45.414522 12762 consensus_queue.cc:237] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.418988 12773 raft_consensus.cc:686] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:45.420024 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:45.420802 12445 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.422885 12773 consensus_queue.cc:237] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.427556 12445 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.431988 12341 catalog_manager.cc:5484] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 2 to 3, leader changed from 49e3300ff51e489889daf9f659d10b13 (127.12.6.1) to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 3 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:45.433082 12341 catalog_manager.cc:5484] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 2 to 3, leader changed from d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3) to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 3 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:45.434041 12341 catalog_manager.cc:5484] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 2 to 3, leader changed from d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3) to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 3 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:45.448983 12767 raft_consensus.cc:979] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: : Instructing follower 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.449398 12767 raft_consensus.cc:1067] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Signalling peer 6bbad1351d304f2da4f413e6522c0bd0 to start an election
I20240503 08:03:45.450675 12522 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
from {username='slave'} at 127.0.0.1:44192
I20240503 08:03:45.451175 12522 raft_consensus.cc:482] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:45.451472 12522 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.456272 12522 raft_consensus.cc:504] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.457918 12522 leader_election.cc:288] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:45.458932 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:45.459017 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:45.459481 12596 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.459555 12445 raft_consensus.cc:3032] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:03:45.459908 12445 raft_consensus.cc:729] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:45.460415 12445 consensus_queue.cc:260] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.461251 12445 raft_consensus.cc:3037] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:03:45.464258 12596 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.465135 12445 raft_consensus.cc:2454] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bbad1351d304f2da4f413e6522c0bd0 in term 3.
I20240503 08:03:45.465404 12485 leader_election.cc:302] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:45.466064 12773 raft_consensus.cc:2781] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:03:45.466416 12773 raft_consensus.cc:686] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:45.467170 12773 consensus_queue.cc:237] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:45.473357 12341 catalog_manager.cc:5484] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 reported cstate change: term changed from 2 to 3, leader changed from 49e3300ff51e489889daf9f659d10b13 (127.12.6.1) to 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2). New cstate: current_term: 3 leader_uuid: "6bbad1351d304f2da4f413e6522c0bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:45.800390 12596 raft_consensus.cc:1261] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.801597 12773 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.810672 12445 raft_consensus.cc:1261] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.812463 12776 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:03:45.837194 12596 raft_consensus.cc:1261] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.838378 12773 consensus_queue.cc:1035] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.846980 12445 raft_consensus.cc:1261] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.848366 12773 consensus_queue.cc:1035] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:03:45.911067 12445 raft_consensus.cc:1261] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.912258 12773 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.920624 12596 raft_consensus.cc:1261] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.921913 12762 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.940068 12445 raft_consensus.cc:1261] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.941185 12770 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.943784 12443 raft_consensus.cc:1261] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.944980 12762 consensus_queue.cc:1035] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.951191 12596 raft_consensus.cc:1261] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.952677 12770 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:45.963464 12594 raft_consensus.cc:1261] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Refusing update from remote peer 6bbad1351d304f2da4f413e6522c0bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:03:45.964854 12773 consensus_queue.cc:1035] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:03:48.312196 12312 auto_leader_rebalancer-test.cc:295] The leader distribution is
I20240503 08:03:48.313046 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:48.320065 12522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "772658b6e692478fae18294150f1f32a"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:42360
I20240503 08:03:48.320883 12522 raft_consensus.cc:595] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Received request to transfer leadership to TS d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:48.323962 12522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:42360
I20240503 08:03:48.324463 12522 raft_consensus.cc:595] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Received request to transfer leadership to TS 49e3300ff51e489889daf9f659d10b13
I20240503 08:03:48.326771 12522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bac0059e531d4156b96e978e5e610408"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:42360
I20240503 08:03:48.327494 12522 raft_consensus.cc:595] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Received request to transfer leadership to TS 49e3300ff51e489889daf9f659d10b13
I20240503 08:03:48.330348 12522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:42360
I20240503 08:03:48.330921 12522 raft_consensus.cc:595] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Received request to transfer leadership to TS d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:48.333559 12522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c"
dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
mode: GRACEFUL
new_leader_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:42360
I20240503 08:03:48.334110 12522 raft_consensus.cc:595] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Received request to transfer leadership to TS 49e3300ff51e489889daf9f659d10b13
I20240503 08:03:48.335233 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 5
I20240503 08:03:48.335672 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:48.381776 12797 raft_consensus.cc:979] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:48.382292 12776 raft_consensus.cc:1067] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Signalling peer d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:48.383711 12596 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "772658b6e692478fae18294150f1f32a"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:55374
I20240503 08:03:48.384167 12596 raft_consensus.cc:482] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:48.384491 12596 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.390388 12596 raft_consensus.cc:504] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.392357 12596 leader_election.cc:288] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 4 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:48.393323 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:48.393519 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:48.394004 12445 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.394105 12522 raft_consensus.cc:3032] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Stepping down as leader of term 3
I20240503 08:03:48.394450 12522 raft_consensus.cc:729] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:48.395200 12522 consensus_queue.cc:260] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.396260 12522 raft_consensus.cc:3037] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.398737 12445 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 4.
I20240503 08:03:48.399925 12559 leader_election.cc:302] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:48.400697 12795 raft_consensus.cc:2781] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 08:03:48.401805 12795 raft_consensus.cc:686] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 4 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:48.401458 12522 raft_consensus.cc:2454] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 4.
I20240503 08:03:48.403255 12795 consensus_queue.cc:237] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.410874 12343 catalog_manager.cc:5484] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 3 to 4, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 4 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:48.424089 12797 raft_consensus.cc:979] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:48.424571 12776 raft_consensus.cc:1067] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Signalling peer 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:48.425889 12445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bac0059e531d4156b96e978e5e610408"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:45768
I20240503 08:03:48.426290 12445 raft_consensus.cc:482] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:48.426517 12445 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.430729 12445 raft_consensus.cc:504] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.432647 12445 leader_election.cc:288] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 4 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:48.433539 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:48.433621 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac0059e531d4156b96e978e5e610408" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:48.434042 12522 raft_consensus.cc:3032] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Stepping down as leader of term 3
I20240503 08:03:48.434105 12596 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.434507 12522 raft_consensus.cc:729] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:48.435297 12522 consensus_queue.cc:260] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.436249 12522 raft_consensus.cc:3037] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.438536 12596 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 4.
I20240503 08:03:48.439738 12410 leader_election.cc:302] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:48.441074 12522 raft_consensus.cc:2454] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 4.
I20240503 08:03:48.441457 12804 raft_consensus.cc:2781] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 08:03:48.442798 12804 raft_consensus.cc:686] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 4 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:48.443629 12804 consensus_queue.cc:237] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.451267 12343 catalog_manager.cc:5484] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 3 to 4, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 4 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:48.589653 12797 raft_consensus.cc:979] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:48.590138 12776 raft_consensus.cc:1067] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Signalling peer 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:48.591709 12445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:45768
I20240503 08:03:48.592357 12445 raft_consensus.cc:482] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:48.592700 12445 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.597009 12445 raft_consensus.cc:504] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.598428 12445 leader_election.cc:288] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 4 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:48.599287 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:48.599279 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e7df304e0a848f787eea813c111b4f3" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:48.600008 12596 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.600064 12522 raft_consensus.cc:3032] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Stepping down as leader of term 3
I20240503 08:03:48.600468 12522 raft_consensus.cc:729] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:48.601276 12522 consensus_queue.cc:260] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.602607 12522 raft_consensus.cc:3037] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:03:48.606451 12596 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 4.
I20240503 08:03:48.607465 12410 leader_election.cc:302] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:48.608222 12804 raft_consensus.cc:2781] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 08:03:48.608636 12804 raft_consensus.cc:686] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 4 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:48.609189 12522 raft_consensus.cc:2454] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 4.
I20240503 08:03:48.609534 12804 consensus_queue.cc:237] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.616586 12343 catalog_manager.cc:5484] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 3 to 4, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 4 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:48.671119 12776 raft_consensus.cc:979] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:48.671597 12797 raft_consensus.cc:1067] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Signalling peer 49e3300ff51e489889daf9f659d10b13 to start an election
I20240503 08:03:48.672973 12445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c"
dest_uuid: "49e3300ff51e489889daf9f659d10b13"
from {username='slave'} at 127.0.0.1:45768
I20240503 08:03:48.673425 12445 raft_consensus.cc:482] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:48.673667 12445 raft_consensus.cc:3037] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:48.677855 12445 raft_consensus.cc:504] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.679364 12445 leader_election.cc:288] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Requested vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:48.680202 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:48.680574 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
I20240503 08:03:48.680958 12522 raft_consensus.cc:3032] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:48.681362 12522 raft_consensus.cc:729] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:48.682070 12596 raft_consensus.cc:3037] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:48.682137 12522 consensus_queue.cc:260] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.683598 12522 raft_consensus.cc:3037] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:48.688670 12596 raft_consensus.cc:2454] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:48.689874 12410 leader_election.cc:302] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:48.689990 12522 raft_consensus.cc:2454] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e3300ff51e489889daf9f659d10b13 in term 2.
I20240503 08:03:48.690518 12804 raft_consensus.cc:2781] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:48.690991 12804 raft_consensus.cc:686] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Becoming Leader. State: Replica: 49e3300ff51e489889daf9f659d10b13, State: Running, Role: LEADER
I20240503 08:03:48.691794 12804 consensus_queue.cc:237] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.697938 12343 catalog_manager.cc:5484] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 reported cstate change: term changed from 1 to 2, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to 49e3300ff51e489889daf9f659d10b13 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "49e3300ff51e489889daf9f659d10b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: UNKNOWN } } }
I20240503 08:03:48.796300 12522 raft_consensus.cc:1261] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:03:48.797559 12795 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:03:48.810479 12445 raft_consensus.cc:1261] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:03:48.812001 12811 consensus_queue.cc:1035] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:03:48.827358 12776 raft_consensus.cc:979] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: : Instructing follower d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:48.827786 12797 raft_consensus.cc:1067] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Signalling peer d318d4aca45244ba9cd5a8cb36cab286 to start an election
I20240503 08:03:48.829038 12596 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0"
dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286"
from {username='slave'} at 127.0.0.1:55374
I20240503 08:03:48.829465 12596 raft_consensus.cc:482] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:03:48.829710 12596 raft_consensus.cc:3037] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:48.833709 12596 raft_consensus.cc:504] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.835191 12596 leader_election.cc:288] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 2 election: Requested vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:48.836100 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49e3300ff51e489889daf9f659d10b13"
I20240503 08:03:48.836194 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d785e3daf54f4bb7b50647703534ecf0" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0"
I20240503 08:03:48.836621 12445 raft_consensus.cc:3037] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:48.836694 12522 raft_consensus.cc:3032] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:03:48.837029 12522 raft_consensus.cc:729] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bbad1351d304f2da4f413e6522c0bd0, State: Running, Role: LEADER
I20240503 08:03:48.837685 12522 consensus_queue.cc:260] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.838639 12522 raft_consensus.cc:3037] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:03:48.841027 12445 raft_consensus.cc:2454] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 2.
I20240503 08:03:48.842023 12559 leader_election.cc:302] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286; no voters:
I20240503 08:03:48.842700 12811 raft_consensus.cc:2781] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:03:48.843217 12811 raft_consensus.cc:686] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Becoming Leader. State: Replica: d318d4aca45244ba9cd5a8cb36cab286, State: Running, Role: LEADER
I20240503 08:03:48.843642 12522 raft_consensus.cc:2454] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d318d4aca45244ba9cd5a8cb36cab286 in term 2.
I20240503 08:03:48.844018 12811 consensus_queue.cc:237] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:48.851073 12341 catalog_manager.cc:5484] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 reported cstate change: term changed from 1 to 2, leader changed from 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2) to d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3). New cstate: current_term: 2 leader_uuid: "d318d4aca45244ba9cd5a8cb36cab286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } health_report { overall_health: HEALTHY } } }
I20240503 08:03:48.889267 12596 raft_consensus.cc:1261] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:03:48.890419 12804 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:03:48.900611 12522 raft_consensus.cc:1261] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:03:48.902159 12804 consensus_queue.cc:1035] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:03:49.029328 12522 raft_consensus.cc:1261] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:03:49.030740 12804 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:03:49.041198 12596 raft_consensus.cc:1261] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:03:49.043005 12804 consensus_queue.cc:1035] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:03:49.175588 12522 raft_consensus.cc:1261] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:49.176951 12804 consensus_queue.cc:1035] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:49.185626 12596 raft_consensus.cc:1261] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Refusing update from remote peer 49e3300ff51e489889daf9f659d10b13: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:49.186998 12805 consensus_queue.cc:1035] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:03:49.295104 12520 raft_consensus.cc:1261] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:49.296582 12811 consensus_queue.cc:1035] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:03:49.305842 12445 raft_consensus.cc:1261] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Refusing update from remote peer d318d4aca45244ba9cd5a8cb36cab286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:03:49.307965 12798 consensus_queue.cc:1035] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:03:49.336532 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:49.341264 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:49.341689 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:50.550992 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:50.667713 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:50.668448 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:50.694411 12811 raft_consensus.cc:482] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49e3300ff51e489889daf9f659d10b13)
I20240503 08:03:50.694999 12811 raft_consensus.cc:504] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:50.697237 12811 leader_election.cc:288] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:50.701859 12443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:50.702780 12805 raft_consensus.cc:482] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bbad1351d304f2da4f413e6522c0bd0)
I20240503 08:03:50.703334 12805 raft_consensus.cc:504] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:50.707751 12805 leader_election.cc:288] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:50.726244 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:50.737587 12795 raft_consensus.cc:482] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6bbad1351d304f2da4f413e6522c0bd0)
I20240503 08:03:50.738121 12795 raft_consensus.cc:504] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:50.740221 12795 leader_election.cc:288] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969)
I20240503 08:03:50.741855 12443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:50.745728 12832 raft_consensus.cc:482] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49e3300ff51e489889daf9f659d10b13)
I20240503 08:03:50.746748 12831 raft_consensus.cc:482] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d318d4aca45244ba9cd5a8cb36cab286)
I20240503 08:03:50.747350 12831 raft_consensus.cc:504] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:50.749981 12831 leader_election.cc:288] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:50.750524 12831 raft_consensus.cc:482] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d318d4aca45244ba9cd5a8cb36cab286)
I20240503 08:03:50.746357 12832 raft_consensus.cc:504] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:50.751055 12831 raft_consensus.cc:504] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e3300ff51e489889daf9f659d10b13" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 46613 } } peers { permanent_uuid: "6bbad1351d304f2da4f413e6522c0bd0" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 32969 } } peers { permanent_uuid: "d318d4aca45244ba9cd5a8cb36cab286" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 42523 } }
I20240503 08:03:50.752489 12832 leader_election.cc:288] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:50.752959 12831 leader_election.cc:288] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49e3300ff51e489889daf9f659d10b13 (127.12.6.1:46613), d318d4aca45244ba9cd5a8cb36cab286 (127.12.6.3:42523)
I20240503 08:03:50.755751 12521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968e24516804ba8aa9c02c264039dc2" candidate_uuid: "49e3300ff51e489889daf9f659d10b13" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:50.756764 12520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:50.757979 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18a893675482c9720556a30c9ed10" candidate_uuid: "d318d4aca45244ba9cd5a8cb36cab286" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6bbad1351d304f2da4f413e6522c0bd0" is_pre_election: true
I20240503 08:03:50.758090 12443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:50.758713 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8474bb05d88f4292a5d2a597cab5042f" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:50.758728 12445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:50.760555 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:50.760798 12443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772658b6e692478fae18294150f1f32a" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49e3300ff51e489889daf9f659d10b13" is_pre_election: true
I20240503 08:03:50.762373 12412 leader_election.cc:302] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49e3300ff51e489889daf9f659d10b13; no voters: 6bbad1351d304f2da4f413e6522c0bd0, d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:50.763280 12805 raft_consensus.cc:2726] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 08:03:50.764022 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dacbaaaa60fd496fa2d2c195e8a2210c" candidate_uuid: "6bbad1351d304f2da4f413e6522c0bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d318d4aca45244ba9cd5a8cb36cab286" is_pre_election: true
I20240503 08:03:50.765625 12485 leader_election.cc:302] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0; no voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:50.765588 12561 leader_election.cc:302] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d318d4aca45244ba9cd5a8cb36cab286; no voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:50.766718 12826 raft_consensus.cc:2726] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240503 08:03:50.766759 12561 leader_election.cc:302] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d318d4aca45244ba9cd5a8cb36cab286; no voters: 49e3300ff51e489889daf9f659d10b13, 6bbad1351d304f2da4f413e6522c0bd0
I20240503 08:03:50.767414 12795 raft_consensus.cc:2726] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20240503 08:03:50.767614 12832 raft_consensus.cc:2726] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 08:03:50.768241 12485 leader_election.cc:302] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0; no voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286
W20240503 08:03:50.768306 12375 debug-util.cc:398] Leaking SignalData structure 0x7b0800061060 after lost signal to thread 12315
I20240503 08:03:50.768821 12832 raft_consensus.cc:2726] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20240503 08:03:50.769045 12375 debug-util.cc:398] Leaking SignalData structure 0x7b0800061740 after lost signal to thread 12378
I20240503 08:03:50.769270 12485 leader_election.cc:302] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bbad1351d304f2da4f413e6522c0bd0; no voters: 49e3300ff51e489889daf9f659d10b13, d318d4aca45244ba9cd5a8cb36cab286
I20240503 08:03:50.769784 12832 raft_consensus.cc:2726] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20240503 08:03:50.769858 12375 debug-util.cc:398] Leaking SignalData structure 0x7b08001e2b40 after lost signal to thread 12471
W20240503 08:03:50.770723 12375 debug-util.cc:398] Leaking SignalData structure 0x7b08001e69c0 after lost signal to thread 12546
W20240503 08:03:50.771417 12375 debug-util.cc:398] Leaking SignalData structure 0x7b080015c9a0 after lost signal to thread 12620
I20240503 08:03:51.669443 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:51.675706 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:51.676119 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:52.676971 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:52.682826 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:52.683277 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:53.684000 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:53.689787 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:53.690269 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:54.691030 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:54.696805 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:54.697209 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:55.697952 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:55.702977 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:55.703531 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:56.704344 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:56.709286 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:56.709753 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:57.710631 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:03:57.715857 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:03:57.716398 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:03:58.744009 12312 auto_leader_rebalancer-test.cc:306] The leader distribution is
I20240503 08:03:58.744688 12312 tablet_server.cc:178] TabletServer@127.12.6.1:0 shutting down...
I20240503 08:03:58.775465 12312 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 08:03:58.776297 12312 tablet_replica.cc:330] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.777437 12312 raft_consensus.cc:2229] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 4 LEADER]: Raft consensus shutting down.
I20240503 08:03:58.778714 12312 raft_consensus.cc:2258] T bac0059e531d4156b96e978e5e610408 P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.783088 12312 tablet_replica.cc:330] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.783751 12312 raft_consensus.cc:2229] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 2 LEADER]: Raft consensus shutting down.
I20240503 08:03:58.784570 12312 raft_consensus.cc:2258] T dacbaaaa60fd496fa2d2c195e8a2210c P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.786381 12312 tablet_replica.cc:330] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.786960 12312 raft_consensus.cc:2229] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.787510 12312 raft_consensus.cc:2258] T 8ce18a893675482c9720556a30c9ed10 P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.789295 12312 tablet_replica.cc:330] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.789817 12312 raft_consensus.cc:2229] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.790361 12312 raft_consensus.cc:2258] T 772658b6e692478fae18294150f1f32a P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.792291 12312 tablet_replica.cc:330] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.792804 12312 raft_consensus.cc:2229] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.793287 12312 raft_consensus.cc:2258] T 3958d27365e7439494639369ef57cd9b P 49e3300ff51e489889daf9f659d10b13 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.795360 12312 tablet_replica.cc:330] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.796042 12312 raft_consensus.cc:2229] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.796645 12312 raft_consensus.cc:2258] T 0968e24516804ba8aa9c02c264039dc2 P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.799068 12312 tablet_replica.cc:330] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.799840 12312 raft_consensus.cc:2229] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.800367 12312 raft_consensus.cc:2258] T 8474bb05d88f4292a5d2a597cab5042f P 49e3300ff51e489889daf9f659d10b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.802799 12312 tablet_replica.cc:330] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.803483 12312 raft_consensus.cc:2229] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.803992 12312 raft_consensus.cc:2258] T d785e3daf54f4bb7b50647703534ecf0 P 49e3300ff51e489889daf9f659d10b13 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.806411 12312 tablet_replica.cc:330] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13: stopping tablet replica
I20240503 08:03:58.807045 12312 raft_consensus.cc:2229] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 4 LEADER]: Raft consensus shutting down.
I20240503 08:03:58.808151 12312 raft_consensus.cc:2258] T 7e7df304e0a848f787eea813c111b4f3 P 49e3300ff51e489889daf9f659d10b13 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.893728 12312 tablet_server.cc:194] TabletServer@127.12.6.1:0 shutdown complete.
I20240503 08:03:58.913282 12312 tablet_server.cc:178] TabletServer@127.12.6.2:0 shutting down...
I20240503 08:03:58.942795 12312 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 08:03:58.943534 12312 tablet_replica.cc:330] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.944190 12312 raft_consensus.cc:2229] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.944658 12312 raft_consensus.cc:2258] T dacbaaaa60fd496fa2d2c195e8a2210c P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.946398 12312 tablet_replica.cc:330] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.946887 12312 raft_consensus.cc:2229] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.947361 12312 raft_consensus.cc:2258] T 8474bb05d88f4292a5d2a597cab5042f P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.949033 12312 tablet_replica.cc:330] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.949483 12312 raft_consensus.cc:2229] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 LEADER]: Raft consensus shutting down.
I20240503 08:03:58.950222 12312 raft_consensus.cc:2258] T 0968e24516804ba8aa9c02c264039dc2 P 6bbad1351d304f2da4f413e6522c0bd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.951934 12312 tablet_replica.cc:330] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.952410 12312 raft_consensus.cc:2229] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.952817 12312 raft_consensus.cc:2258] T d785e3daf54f4bb7b50647703534ecf0 P 6bbad1351d304f2da4f413e6522c0bd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.954423 12312 tablet_replica.cc:330] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.954883 12312 raft_consensus.cc:2229] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.955338 12312 raft_consensus.cc:2258] T 7e7df304e0a848f787eea813c111b4f3 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.956885 12312 tablet_replica.cc:330] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.957355 12312 raft_consensus.cc:2229] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:03:58.958129 12312 raft_consensus.cc:2258] T 8ce18a893675482c9720556a30c9ed10 P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.959758 12312 tablet_replica.cc:330] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.960208 12312 raft_consensus.cc:2229] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:03:58.960960 12312 raft_consensus.cc:2258] T 3958d27365e7439494639369ef57cd9b P 6bbad1351d304f2da4f413e6522c0bd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.962474 12312 tablet_replica.cc:330] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.962921 12312 raft_consensus.cc:2229] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.963357 12312 raft_consensus.cc:2258] T 772658b6e692478fae18294150f1f32a P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:58.964876 12312 tablet_replica.cc:330] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0: stopping tablet replica
I20240503 08:03:58.965335 12312 raft_consensus.cc:2229] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:58.965723 12312 raft_consensus.cc:2258] T bac0059e531d4156b96e978e5e610408 P 6bbad1351d304f2da4f413e6522c0bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
W20240503 08:03:58.990990 12561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.6.2:32969: connect: Connection refused (error 111)
I20240503 08:03:58.994462 12312 tablet_server.cc:194] TabletServer@127.12.6.2:0 shutdown complete.
W20240503 08:03:58.994751 12561 consensus_peers.cc:485] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 -> Peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Couldn't send request to peer 6bbad1351d304f2da4f413e6522c0bd0. Status: Network error: Client connection negotiation failed: client connection to 127.12.6.2:32969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 08:03:59.011085 12312 tablet_server.cc:178] TabletServer@127.12.6.3:0 shutting down...
W20240503 08:03:59.039203 12561 consensus_peers.cc:485] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 -> Peer 6bbad1351d304f2da4f413e6522c0bd0 (127.12.6.2:32969): Couldn't send request to peer 6bbad1351d304f2da4f413e6522c0bd0. Status: Network error: Client connection negotiation failed: client connection to 127.12.6.2:32969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 08:03:59.040907 12312 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 08:03:59.041505 12312 tablet_replica.cc:330] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.042105 12312 raft_consensus.cc:2229] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:59.042693 12312 raft_consensus.cc:2258] T bac0059e531d4156b96e978e5e610408 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.044538 12312 tablet_replica.cc:330] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.045029 12312 raft_consensus.cc:2229] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 LEADER]: Raft consensus shutting down.
I20240503 08:03:59.045745 12312 raft_consensus.cc:2258] T d785e3daf54f4bb7b50647703534ecf0 P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.047421 12312 tablet_replica.cc:330] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.047916 12312 raft_consensus.cc:2229] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:59.048306 12312 raft_consensus.cc:2258] T 7e7df304e0a848f787eea813c111b4f3 P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.049832 12312 tablet_replica.cc:330] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.050300 12312 raft_consensus.cc:2229] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:59.050705 12312 raft_consensus.cc:2258] T 3958d27365e7439494639369ef57cd9b P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.052548 12312 tablet_replica.cc:330] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.053122 12312 raft_consensus.cc:2229] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:59.053546 12312 raft_consensus.cc:2258] T 8ce18a893675482c9720556a30c9ed10 P d318d4aca45244ba9cd5a8cb36cab286 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.055310 12312 tablet_replica.cc:330] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.055797 12312 raft_consensus.cc:2229] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 4 LEADER]: Raft consensus shutting down.
I20240503 08:03:59.056502 12312 raft_consensus.cc:2258] T 772658b6e692478fae18294150f1f32a P d318d4aca45244ba9cd5a8cb36cab286 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.058111 12312 tablet_replica.cc:330] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.058593 12312 raft_consensus.cc:2229] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:59.058933 12312 raft_consensus.cc:2258] T dacbaaaa60fd496fa2d2c195e8a2210c P d318d4aca45244ba9cd5a8cb36cab286 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.060489 12312 tablet_replica.cc:330] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.061033 12312 raft_consensus.cc:2229] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:03:59.061410 12312 raft_consensus.cc:2258] T 0968e24516804ba8aa9c02c264039dc2 P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.062983 12312 tablet_replica.cc:330] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286: stopping tablet replica
I20240503 08:03:59.063408 12312 raft_consensus.cc:2229] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 1 LEADER]: Raft consensus shutting down.
I20240503 08:03:59.064131 12312 raft_consensus.cc:2258] T 8474bb05d88f4292a5d2a597cab5042f P d318d4aca45244ba9cd5a8cb36cab286 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.091185 12312 tablet_server.cc:194] TabletServer@127.12.6.3:0 shutdown complete.
I20240503 08:03:59.108003 12312 master.cc:537] Master@127.12.6.62:36233 shutting down...
I20240503 08:03:59.322018 12312 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 1 LEADER]: Raft consensus shutting down.
I20240503 08:03:59.322678 12312 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:03:59.323004 12312 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7d4c9555562445cd8fefee7c761c3bbc: stopping tablet replica
I20240503 08:03:59.357826 12312 master.cc:552] Master@127.12.6.62:36233 shutdown complete.
49e3300ff51e489889daf9f659d10b13 4
6bbad1351d304f2da4f413e6522c0bd0 4
d318d4aca45244ba9cd5a8cb36cab286 1
49e3300ff51e489889daf9f659d10b13 3
6bbad1351d304f2da4f413e6522c0bd0 3
d318d4aca45244ba9cd5a8cb36cab286 3
49e3300ff51e489889daf9f659d10b13 0
6bbad1351d304f2da4f413e6522c0bd0 8
d318d4aca45244ba9cd5a8cb36cab286 1
49e3300ff51e489889daf9f659d10b13 3
6bbad1351d304f2da4f413e6522c0bd0 3
d318d4aca45244ba9cd5a8cb36cab286 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (31628 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20240503 08:03:59.402689 12312 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.6.62:38357
I20240503 08:03:59.403838 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:59.410522 12839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.411574 12840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.412501 12842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:59.413390 12312 server_base.cc:1034] running on GCE node
I20240503 08:03:59.414327 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:59.414510 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:59.414670 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723439414650 us; error 0 us; skew 500 ppm
I20240503 08:03:59.415246 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:59.422266 12312 webserver.cc:460] Webserver started at http://127.12.6.62:46175/ using document root <none> and password file <none>
I20240503 08:03:59.422807 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:59.422997 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:59.423300 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:59.424489 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/master-0-root/instance:
uuid: "70ddc3ee2b3341038f11469260b400bb"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.429075 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240503 08:03:59.432329 12847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.433130 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240503 08:03:59.433413 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/master-0-root
uuid: "70ddc3ee2b3341038f11469260b400bb"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.433727 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:59.453327 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:59.454574 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:59.496057 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.62:38357
I20240503 08:03:59.496127 12898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.62:38357 every 8 connection(s)
I20240503 08:03:59.500314 12899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:03:59.511152 12899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb: Bootstrap starting.
I20240503 08:03:59.515748 12899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb: Neither blocks nor log segments found. Creating new log.
I20240503 08:03:59.520002 12899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb: No bootstrap required, opened a new log
I20240503 08:03:59.522192 12899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ddc3ee2b3341038f11469260b400bb" member_type: VOTER }
I20240503 08:03:59.522661 12899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:03:59.522888 12899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ddc3ee2b3341038f11469260b400bb, State: Initialized, Role: FOLLOWER
I20240503 08:03:59.523571 12899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ddc3ee2b3341038f11469260b400bb" member_type: VOTER }
I20240503 08:03:59.524068 12899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 08:03:59.524268 12899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 08:03:59.524533 12899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:03:59.533174 12899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ddc3ee2b3341038f11469260b400bb" member_type: VOTER }
I20240503 08:03:59.533756 12899 leader_election.cc:302] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70ddc3ee2b3341038f11469260b400bb; no voters:
I20240503 08:03:59.534982 12899 leader_election.cc:288] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 08:03:59.535578 12902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:03:59.536880 12902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 1 LEADER]: Becoming Leader. State: Replica: 70ddc3ee2b3341038f11469260b400bb, State: Running, Role: LEADER
I20240503 08:03:59.537601 12902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ddc3ee2b3341038f11469260b400bb" member_type: VOTER }
I20240503 08:03:59.538012 12899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [sys.catalog]: configured and running, proceeding with master startup.
I20240503 08:03:59.542248 12903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70ddc3ee2b3341038f11469260b400bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ddc3ee2b3341038f11469260b400bb" member_type: VOTER } }
I20240503 08:03:59.542626 12904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70ddc3ee2b3341038f11469260b400bb. Latest consensus state: current_term: 1 leader_uuid: "70ddc3ee2b3341038f11469260b400bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ddc3ee2b3341038f11469260b400bb" member_type: VOTER } }
I20240503 08:03:59.543282 12904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [sys.catalog]: This master's current role is: LEADER
I20240503 08:03:59.543289 12903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [sys.catalog]: This master's current role is: LEADER
I20240503 08:03:59.545816 12908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 08:03:59.551604 12908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 08:03:59.557327 12312 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240503 08:03:59.560809 12908 catalog_manager.cc:1347] Generated new cluster ID: e981b3dac57c479283d431f82bc6c1b9
I20240503 08:03:59.561102 12908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 08:03:59.588609 12908 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 08:03:59.589977 12908 catalog_manager.cc:1504] Loading token signing keys...
I20240503 08:03:59.602483 12908 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb: Generated new TSK 0
I20240503 08:03:59.603065 12908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 08:03:59.624178 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:59.631326 12920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.632056 12921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.633198 12923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:59.634307 12312 server_base.cc:1034] running on GCE node
I20240503 08:03:59.635380 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:59.635571 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:59.635710 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723439635696 us; error 0 us; skew 500 ppm
I20240503 08:03:59.636282 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:59.638787 12312 webserver.cc:460] Webserver started at http://127.12.6.1:45617/ using document root <none> and password file <none>
I20240503 08:03:59.639343 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:59.639505 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:59.639827 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:59.640946 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-0-root/instance:
uuid: "87a73f1536014d609105112116c0e957"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.645512 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20240503 08:03:59.649005 12928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.649945 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240503 08:03:59.650226 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-0-root
uuid: "87a73f1536014d609105112116c0e957"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.650516 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:59.680709 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:59.681922 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:59.683465 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:03:59.685789 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:03:59.686002 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.686233 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:03:59.686411 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.728063 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.1:34959
I20240503 08:03:59.728170 12990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.1:34959 every 8 connection(s)
I20240503 08:03:59.732599 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:59.741407 12995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.742589 12996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.743786 12998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:59.745108 12312 server_base.cc:1034] running on GCE node
I20240503 08:03:59.745839 12991 heartbeater.cc:347] Connected to a master server at 127.12.6.62:38357
I20240503 08:03:59.746173 12991 heartbeater.cc:464] Registering TS with master...
I20240503 08:03:59.746623 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:59.746975 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:59.747225 12991 heartbeater.cc:511] Master 127.12.6.62:38357 requested a full tablet report, sending...
I20240503 08:03:59.747224 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723439747203 us; error 0 us; skew 500 ppm
I20240503 08:03:59.748307 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:59.750013 12864 ts_manager.cc:194] Registered new tserver with Master: 87a73f1536014d609105112116c0e957 (127.12.6.1:34959)
I20240503 08:03:59.751466 12312 webserver.cc:460] Webserver started at http://127.12.6.2:36107/ using document root <none> and password file <none>
I20240503 08:03:59.752163 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:59.752405 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:59.752590 12864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37840
I20240503 08:03:59.752820 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:59.754278 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-1-root/instance:
uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.759063 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20240503 08:03:59.762859 13003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.763689 12312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240503 08:03:59.763976 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-1-root
uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.764263 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:59.784236 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:59.785461 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:59.786844 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:03:59.789114 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:03:59.789326 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.789556 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:03:59.789711 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.829360 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.2:33877
I20240503 08:03:59.829495 13065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.2:33877 every 8 connection(s)
I20240503 08:03:59.834240 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:03:59.841796 13069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:03:59.844278 13070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:59.845876 13066 heartbeater.cc:347] Connected to a master server at 127.12.6.62:38357
I20240503 08:03:59.846246 13066 heartbeater.cc:464] Registering TS with master...
I20240503 08:03:59.846266 12312 server_base.cc:1034] running on GCE node
W20240503 08:03:59.847278 13072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:03:59.847354 13066 heartbeater.cc:511] Master 127.12.6.62:38357 requested a full tablet report, sending...
I20240503 08:03:59.848274 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:03:59.848479 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:03:59.848656 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723439848638 us; error 0 us; skew 500 ppm
I20240503 08:03:59.849362 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:03:59.849658 12864 ts_manager.cc:194] Registered new tserver with Master: 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:03:59.851539 12864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37850
I20240503 08:03:59.852418 12312 webserver.cc:460] Webserver started at http://127.12.6.3:35765/ using document root <none> and password file <none>
I20240503 08:03:59.853111 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:03:59.853370 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:03:59.853706 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:03:59.854970 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-2-root/instance:
uuid: "6cfd1f6d1cef47349921f57394a0212c"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.859513 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240503 08:03:59.862749 13077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.863538 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240503 08:03:59.863828 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-2-root
uuid: "6cfd1f6d1cef47349921f57394a0212c"
format_stamp: "Formatted at 2024-05-03 08:03:59 on dist-test-slave-6qcz"
I20240503 08:03:59.864110 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714723407607891-12312-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:03:59.892104 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:03:59.893271 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:03:59.894727 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:03:59.897047 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:03:59.897259 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.897495 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:03:59.897655 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:03:59.940536 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.3:33615
I20240503 08:03:59.940807 13139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.3:33615 every 8 connection(s)
I20240503 08:03:59.954532 13140 heartbeater.cc:347] Connected to a master server at 127.12.6.62:38357
I20240503 08:03:59.955051 13140 heartbeater.cc:464] Registering TS with master...
I20240503 08:03:59.955854 13140 heartbeater.cc:511] Master 127.12.6.62:38357 requested a full tablet report, sending...
I20240503 08:03:59.957890 12864 ts_manager.cc:194] Registered new tserver with Master: 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:03:59.958918 12312 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015199243s
I20240503 08:03:59.959285 12864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37860
I20240503 08:03:59.961720 12312 test_util.cc:268] Using random seed: 696706657
I20240503 08:03:59.985625 12864 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37872:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 08:03:59.988667 12864 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 08:04:00.062644 13105 tablet_service.cc:1467] Processing CreateTablet for tablet b05bc22dbb28441cb642e218aa692971 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240503 08:04:00.063514 13103 tablet_service.cc:1467] Processing CreateTablet for tablet c088bd75653841da80100a14604827b9 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240503 08:04:00.064328 13105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05bc22dbb28441cb642e218aa692971. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.064965 13103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c088bd75653841da80100a14604827b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.066074 13104 tablet_service.cc:1467] Processing CreateTablet for tablet 916c02f4a324485d883ead3fdd81f7e1 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240503 08:04:00.067489 13104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916c02f4a324485d883ead3fdd81f7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.083106 13155 tablet_bootstrap.cc:492] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.085850 13105 tablet_service.cc:1467] Processing CreateTablet for tablet 7396a6e047f14194afdfaa32186a8858 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240503 08:04:00.086405 13102 tablet_service.cc:1467] Processing CreateTablet for tablet 3f33b65ff9684d38a7671f1f88e0c190 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240503 08:04:00.087569 13105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7396a6e047f14194afdfaa32186a8858. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.089773 13101 tablet_service.cc:1467] Processing CreateTablet for tablet 5c013c695821420597e5385fd5a679a1 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240503 08:04:00.094218 13100 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa48fa501614e9d88dababf62e20b36 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240503 08:04:00.095595 13104 tablet_service.cc:1467] Processing CreateTablet for tablet 72e6c2a47f6243ae9d98439b9cb5ff52 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240503 08:04:00.096149 13100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa48fa501614e9d88dababf62e20b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.096933 13104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e6c2a47f6243ae9d98439b9cb5ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.097642 13102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f33b65ff9684d38a7671f1f88e0c190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.099489 13101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c013c695821420597e5385fd5a679a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.105234 13099 tablet_service.cc:1467] Processing CreateTablet for tablet a484222860f74f7284dbc7fdec58c36f (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240503 08:04:00.102921 13155 tablet_bootstrap.cc:654] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.106781 13099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a484222860f74f7284dbc7fdec58c36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.111992 13103 tablet_service.cc:1467] Processing CreateTablet for tablet 05c22dbc72b046788317d845ec3ef7b7 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240503 08:04:00.113361 13103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c22dbc72b046788317d845ec3ef7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.121672 12956 tablet_service.cc:1467] Processing CreateTablet for tablet b05bc22dbb28441cb642e218aa692971 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240503 08:04:00.123091 12956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05bc22dbb28441cb642e218aa692971. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.137557 13155 tablet_bootstrap.cc:492] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.138244 13155 ts_tablet_manager.cc:1367] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.056s user 0.021s sys 0.000s
I20240503 08:04:00.124516 12955 tablet_service.cc:1467] Processing CreateTablet for tablet 916c02f4a324485d883ead3fdd81f7e1 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240503 08:04:00.140623 12955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916c02f4a324485d883ead3fdd81f7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.141403 13155 raft_consensus.cc:348] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.125162 13031 tablet_service.cc:1467] Processing CreateTablet for tablet b05bc22dbb28441cb642e218aa692971 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240503 08:04:00.125504 13030 tablet_service.cc:1467] Processing CreateTablet for tablet 916c02f4a324485d883ead3fdd81f7e1 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240503 08:04:00.126953 12953 tablet_service.cc:1467] Processing CreateTablet for tablet 05c22dbc72b046788317d845ec3ef7b7 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240503 08:04:00.143100 13031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05bc22dbb28441cb642e218aa692971. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.143755 12953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c22dbc72b046788317d845ec3ef7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.143819 13030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916c02f4a324485d883ead3fdd81f7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.127580 13028 tablet_service.cc:1467] Processing CreateTablet for tablet 05c22dbc72b046788317d845ec3ef7b7 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240503 08:04:00.148768 13028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c22dbc72b046788317d845ec3ef7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.128149 12952 tablet_service.cc:1467] Processing CreateTablet for tablet 7396a6e047f14194afdfaa32186a8858 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240503 08:04:00.150446 12952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7396a6e047f14194afdfaa32186a8858. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.129379 12951 tablet_service.cc:1467] Processing CreateTablet for tablet 3f33b65ff9684d38a7671f1f88e0c190 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240503 08:04:00.151636 13155 raft_consensus.cc:374] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.152762 13155 raft_consensus.cc:729] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.129946 13026 tablet_service.cc:1467] Processing CreateTablet for tablet 3f33b65ff9684d38a7671f1f88e0c190 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240503 08:04:00.162122 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f33b65ff9684d38a7671f1f88e0c190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.130582 12950 tablet_service.cc:1467] Processing CreateTablet for tablet 5c013c695821420597e5385fd5a679a1 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240503 08:04:00.131170 13025 tablet_service.cc:1467] Processing CreateTablet for tablet 5c013c695821420597e5385fd5a679a1 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240503 08:04:00.168859 13025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c013c695821420597e5385fd5a679a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.131826 12949 tablet_service.cc:1467] Processing CreateTablet for tablet 72e6c2a47f6243ae9d98439b9cb5ff52 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240503 08:04:00.132401 13024 tablet_service.cc:1467] Processing CreateTablet for tablet 72e6c2a47f6243ae9d98439b9cb5ff52 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240503 08:04:00.133042 12948 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa48fa501614e9d88dababf62e20b36 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240503 08:04:00.133615 13023 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa48fa501614e9d88dababf62e20b36 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240503 08:04:00.134279 12947 tablet_service.cc:1467] Processing CreateTablet for tablet a484222860f74f7284dbc7fdec58c36f (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240503 08:04:00.134855 13022 tablet_service.cc:1467] Processing CreateTablet for tablet a484222860f74f7284dbc7fdec58c36f (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240503 08:04:00.125727 12954 tablet_service.cc:1467] Processing CreateTablet for tablet c088bd75653841da80100a14604827b9 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240503 08:04:00.128775 13027 tablet_service.cc:1467] Processing CreateTablet for tablet 7396a6e047f14194afdfaa32186a8858 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240503 08:04:00.151810 12951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f33b65ff9684d38a7671f1f88e0c190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.126356 13029 tablet_service.cc:1467] Processing CreateTablet for tablet c088bd75653841da80100a14604827b9 (DEFAULT_TABLE table=test-workload [id=b0b11310c9ae4b4694a29de0b5b55095]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240503 08:04:00.175519 13024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e6c2a47f6243ae9d98439b9cb5ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.176018 12954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c088bd75653841da80100a14604827b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.179562 13022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a484222860f74f7284dbc7fdec58c36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.179697 12948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa48fa501614e9d88dababf62e20b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.183569 12947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a484222860f74f7284dbc7fdec58c36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.187546 12949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e6c2a47f6243ae9d98439b9cb5ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.191751 13023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa48fa501614e9d88dababf62e20b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.192333 12950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c013c695821420597e5385fd5a679a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.200819 13029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c088bd75653841da80100a14604827b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.203629 13027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7396a6e047f14194afdfaa32186a8858. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:00.235610 13155 consensus_queue.cc:260] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.257158 13140 heartbeater.cc:503] Master 127.12.6.62:38357 was elected leader, sending a full tablet report...
I20240503 08:04:00.280782 13155 ts_tablet_manager.cc:1398] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.142s user 0.008s sys 0.000s
I20240503 08:04:00.282320 13155 tablet_bootstrap.cc:492] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.324329 13158 tablet_bootstrap.cc:492] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.330047 13158 tablet_bootstrap.cc:654] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.330855 13155 tablet_bootstrap.cc:654] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.332284 13157 tablet_bootstrap.cc:492] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.347101 13157 tablet_bootstrap.cc:654] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.348587 13158 tablet_bootstrap.cc:492] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.348614 13155 tablet_bootstrap.cc:492] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.349237 13155 ts_tablet_manager.cc:1367] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.005s
I20240503 08:04:00.349237 13158 ts_tablet_manager.cc:1367] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.006s
I20240503 08:04:00.351858 13157 tablet_bootstrap.cc:492] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.352322 13157 ts_tablet_manager.cc:1367] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.004s
I20240503 08:04:00.352483 13155 raft_consensus.cc:348] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.352610 13158 raft_consensus.cc:348] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.353312 13155 raft_consensus.cc:374] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.353379 13158 raft_consensus.cc:374] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.353710 13155 raft_consensus.cc:729] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.353776 13158 raft_consensus.cc:729] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.354470 13157 raft_consensus.cc:348] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.355190 13157 raft_consensus.cc:374] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.354514 13155 consensus_queue.cc:260] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.354514 13158 consensus_queue.cc:260] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.355468 13157 raft_consensus.cc:729] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.356391 13157 consensus_queue.cc:260] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.357776 13155 ts_tablet_manager.cc:1398] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20240503 08:04:00.359091 13155 tablet_bootstrap.cc:492] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.367529 13155 tablet_bootstrap.cc:654] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.372298 13066 heartbeater.cc:503] Master 127.12.6.62:38357 was elected leader, sending a full tablet report...
I20240503 08:04:00.372799 13158 ts_tablet_manager.cc:1398] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.023s user 0.005s sys 0.016s
I20240503 08:04:00.373728 13158 tablet_bootstrap.cc:492] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.379796 13158 tablet_bootstrap.cc:654] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.379721 12991 heartbeater.cc:503] Master 127.12.6.62:38357 was elected leader, sending a full tablet report...
I20240503 08:04:00.380206 13157 ts_tablet_manager.cc:1398] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.028s user 0.018s sys 0.009s
I20240503 08:04:00.381150 13157 tablet_bootstrap.cc:492] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.387322 13157 tablet_bootstrap.cc:654] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.391816 13158 tablet_bootstrap.cc:492] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.392356 13158 ts_tablet_manager.cc:1367] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.019s user 0.017s sys 0.000s
I20240503 08:04:00.395145 13158 raft_consensus.cc:348] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.395969 13158 raft_consensus.cc:374] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.396277 13158 raft_consensus.cc:729] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.396983 13158 consensus_queue.cc:260] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.400959 13158 ts_tablet_manager.cc:1398] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:00.401954 13158 tablet_bootstrap.cc:492] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.407760 13158 tablet_bootstrap.cc:654] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.409485 13155 tablet_bootstrap.cc:492] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.410007 13155 ts_tablet_manager.cc:1367] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.051s user 0.000s sys 0.013s
I20240503 08:04:00.413642 13155 raft_consensus.cc:348] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.414688 13155 raft_consensus.cc:374] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.415082 13155 raft_consensus.cc:729] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.418169 13157 tablet_bootstrap.cc:492] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.417753 13155 consensus_queue.cc:260] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.418668 13157 ts_tablet_manager.cc:1367] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.021s
I20240503 08:04:00.421547 13157 raft_consensus.cc:348] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.422300 13157 raft_consensus.cc:374] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.422596 13157 raft_consensus.cc:729] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.422935 13164 raft_consensus.cc:482] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.423583 13155 ts_tablet_manager.cc:1398] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.013s user 0.010s sys 0.001s
I20240503 08:04:00.423362 13157 consensus_queue.cc:260] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.423454 13164 raft_consensus.cc:504] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.424646 13155 tablet_bootstrap.cc:492] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.431267 13164 leader_election.cc:288] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.433099 13157 ts_tablet_manager.cc:1398] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.014s user 0.005s sys 0.009s
I20240503 08:04:00.434104 13157 tablet_bootstrap.cc:492] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.438913 13158 tablet_bootstrap.cc:492] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.439648 13155 tablet_bootstrap.cc:654] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.440652 13157 tablet_bootstrap.cc:654] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.441399 13158 ts_tablet_manager.cc:1367] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.040s user 0.019s sys 0.003s
I20240503 08:04:00.448460 13158 raft_consensus.cc:348] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.449456 13158 raft_consensus.cc:374] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.449777 13158 raft_consensus.cc:729] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.455205 13158 consensus_queue.cc:260] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.458348 13158 ts_tablet_manager.cc:1398] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20240503 08:04:00.459764 13158 tablet_bootstrap.cc:492] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.468614 13155 tablet_bootstrap.cc:492] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.469187 13155 ts_tablet_manager.cc:1367] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.045s user 0.018s sys 0.006s
I20240503 08:04:00.472644 13157 tablet_bootstrap.cc:492] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.472487 13155 raft_consensus.cc:348] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.473338 13155 raft_consensus.cc:374] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.473392 13157 ts_tablet_manager.cc:1367] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.016s
I20240503 08:04:00.473753 13155 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.474568 13155 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.477149 13155 ts_tablet_manager.cc:1398] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20240503 08:04:00.476817 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c22dbc72b046788317d845ec3ef7b7" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:00.476729 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c22dbc72b046788317d845ec3ef7b7" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.478355 13155 tablet_bootstrap.cc:492] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.478791 13041 raft_consensus.cc:2454] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 0.
I20240503 08:04:00.479889 13157 raft_consensus.cc:348] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.480275 12931 leader_election.cc:302] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:00.481160 13157 raft_consensus.cc:374] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.481516 13164 raft_consensus.cc:2781] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.481966 13164 raft_consensus.cc:482] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.481853 13157 raft_consensus.cc:729] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.482547 13164 raft_consensus.cc:3037] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.483320 13157 consensus_queue.cc:260] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.484848 13155 tablet_bootstrap.cc:654] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.485772 13157 ts_tablet_manager.cc:1398] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20240503 08:04:00.486831 13157 tablet_bootstrap.cc:492] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
W20240503 08:04:00.490621 12932 leader_election.cc:341] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.493885 13157 tablet_bootstrap.cc:654] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.496234 13164 raft_consensus.cc:504] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.498219 13164 leader_election.cc:288] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.499706 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c22dbc72b046788317d845ec3ef7b7" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:00.500329 13155 tablet_bootstrap.cc:492] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.500955 13155 ts_tablet_manager.cc:1367] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.001s
W20240503 08:04:00.501241 12932 leader_election.cc:341] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.501525 13158 tablet_bootstrap.cc:654] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.503100 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c22dbc72b046788317d845ec3ef7b7" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.503808 13041 raft_consensus.cc:3037] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.504276 13155 raft_consensus.cc:348] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.505026 13155 raft_consensus.cc:374] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.505442 13157 tablet_bootstrap.cc:492] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.505352 13155 raft_consensus.cc:729] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.505950 13157 ts_tablet_manager.cc:1367] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.002s
I20240503 08:04:00.506140 13155 consensus_queue.cc:260] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.508634 13155 ts_tablet_manager.cc:1398] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20240503 08:04:00.509622 13155 tablet_bootstrap.cc:492] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.510856 13157 raft_consensus.cc:348] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.511871 13157 raft_consensus.cc:374] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.512190 13157 raft_consensus.cc:729] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.512986 13157 consensus_queue.cc:260] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.515713 13157 ts_tablet_manager.cc:1398] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.009s user 0.004s sys 0.005s
I20240503 08:04:00.516075 13155 tablet_bootstrap.cc:654] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.516707 13157 tablet_bootstrap.cc:492] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.520364 13159 raft_consensus.cc:482] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.521114 13159 raft_consensus.cc:504] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.526886 13159 leader_election.cc:288] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.539019 13157 tablet_bootstrap.cc:654] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:00.552385 12912 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:00.556550 13159 raft_consensus.cc:482] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.557315 13159 raft_consensus.cc:504] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.560424 13159 leader_election.cc:288] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.563232 13164 raft_consensus.cc:482] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.563992 13164 raft_consensus.cc:504] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.565528 13159 raft_consensus.cc:482] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.566256 13159 raft_consensus.cc:504] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.568379 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.569485 13159 leader_election.cc:288] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
W20240503 08:04:00.571208 12931 leader_election.cc:341] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.573055 13041 raft_consensus.cc:2454] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 1.
I20240503 08:04:00.574105 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:00.582317 13115 raft_consensus.cc:2454] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 0.
I20240503 08:04:00.582878 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.583858 13040 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.574822 13164 leader_election.cc:288] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.575549 13157 tablet_bootstrap.cc:492] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.580935 13038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.585774 12932 leader_election.cc:302] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:00.586397 13038 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.581177 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.586998 13164 raft_consensus.cc:2781] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.587530 13164 raft_consensus.cc:482] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.588076 13164 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.588656 13080 leader_election.cc:302] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:00.590054 13159 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.590690 13159 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.593989 13159 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.594792 13159 raft_consensus.cc:2781] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.586416 13157 ts_tablet_manager.cc:1367] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.070s user 0.011s sys 0.004s
I20240503 08:04:00.598608 13159 raft_consensus.cc:482] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.599294 13159 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.600054 12931 leader_election.cc:302] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters: 6cfd1f6d1cef47349921f57394a0212c
I20240503 08:04:00.601965 13169 raft_consensus.cc:2781] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.612886 13080 leader_election.cc:302] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
W20240503 08:04:00.614274 13080 leader_election.cc:341] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.598610 13164 raft_consensus.cc:504] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.616624 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
W20240503 08:04:00.623037 13080 leader_election.cc:341] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.623281 13164 leader_election.cc:288] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.619868 13181 raft_consensus.cc:2781] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.619659 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.620824 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:00.625187 13115 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.627887 13159 raft_consensus.cc:504] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.631390 13159 leader_election.cc:288] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
W20240503 08:04:00.632402 12931 leader_election.cc:341] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.633702 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.634618 13039 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.639328 13181 raft_consensus.cc:482] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.640146 13181 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.645335 13039 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.647071 13080 leader_election.cc:302] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:00.647735 13169 raft_consensus.cc:686] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:00.648247 13115 raft_consensus.cc:2454] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 1.
I20240503 08:04:00.648661 13159 raft_consensus.cc:2781] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.648958 13169 consensus_queue.cc:237] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.649477 12932 leader_election.cc:302] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:00.650962 13159 raft_consensus.cc:686] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:00.652043 13159 consensus_queue.cc:237] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.652693 13158 tablet_bootstrap.cc:492] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.653214 13158 ts_tablet_manager.cc:1367] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.194s user 0.055s sys 0.095s
I20240503 08:04:00.653784 13182 raft_consensus.cc:2781] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.655920 13158 raft_consensus.cc:348] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.652176 13157 raft_consensus.cc:348] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.657234 13158 raft_consensus.cc:374] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.657301 13157 raft_consensus.cc:374] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.657784 13158 raft_consensus.cc:729] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.657817 13157 raft_consensus.cc:729] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.658779 13158 consensus_queue.cc:260] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.659022 13157 consensus_queue.cc:260] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.660760 13155 tablet_bootstrap.cc:492] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.661208 13155 ts_tablet_manager.cc:1367] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.152s user 0.069s sys 0.067s
I20240503 08:04:00.661942 13157 ts_tablet_manager.cc:1398] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.063s user 0.007s sys 0.000s
I20240503 08:04:00.671989 13157 tablet_bootstrap.cc:492] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.663573 13158 ts_tablet_manager.cc:1398] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20240503 08:04:00.667312 12966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.676339 12966 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.676573 13158 tablet_bootstrap.cc:492] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.676780 13157 tablet_bootstrap.cc:654] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.676189 12862 catalog_manager.cc:5484] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 1 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:00.667716 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.668529 12964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.679003 12964 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.669217 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:00.680394 12962 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.680920 12864 catalog_manager.cc:5484] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:00.669776 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.684118 13158 tablet_bootstrap.cc:654] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.664310 13155 raft_consensus.cc:348] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.685122 13155 raft_consensus.cc:374] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.685425 13155 raft_consensus.cc:729] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.686223 13155 consensus_queue.cc:260] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.688817 13155 ts_tablet_manager.cc:1398] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
W20240503 08:04:00.689625 13081 leader_election.cc:341] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 87a73f1536014d609105112116c0e957 (127.12.6.1:34959): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.690132 13081 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957
I20240503 08:04:00.691202 13159 raft_consensus.cc:2726] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:00.695789 13159 raft_consensus.cc:482] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.696276 13159 raft_consensus.cc:504] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.698314 13159 leader_election.cc:288] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.701885 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a484222860f74f7284dbc7fdec58c36f" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.663813 13182 raft_consensus.cc:686] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
W20240503 08:04:00.703337 13080 leader_election.cc:341] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.704118 13182 consensus_queue.cc:237] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.704975 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a484222860f74f7284dbc7fdec58c36f" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.702143 12962 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.664441 13181 raft_consensus.cc:504] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.705627 12963 raft_consensus.cc:2454] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.707501 13181 leader_election.cc:288] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.689807 13155 tablet_bootstrap.cc:492] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.708529 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.709136 13039 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.715270 13081 leader_election.cc:302] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:00.717465 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:00.718101 12963 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.718446 13155 tablet_bootstrap.cc:654] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.722016 13181 raft_consensus.cc:2781] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.724026 13181 raft_consensus.cc:482] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.724449 13181 raft_consensus.cc:3037] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.726497 12963 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.723552 13039 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.728119 13080 leader_election.cc:302] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:00.729000 13159 raft_consensus.cc:2781] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.729501 13159 raft_consensus.cc:686] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:00.730576 13159 consensus_queue.cc:237] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:00.737179 13157 tablet_bootstrap.cc:492] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.737776 13157 ts_tablet_manager.cc:1367] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.066s user 0.022s sys 0.040s
I20240503 08:04:00.737179 13158 tablet_bootstrap.cc:492] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.739113 13158 ts_tablet_manager.cc:1367] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.063s user 0.033s sys 0.028s
I20240503 08:04:00.740967 13157 raft_consensus.cc:348] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.741797 13157 raft_consensus.cc:374] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.741598 13081 leader_election.cc:302] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957
I20240503 08:04:00.742197 13157 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.741925 13158 raft_consensus.cc:348] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.742682 13158 raft_consensus.cc:374] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.742822 13183 raft_consensus.cc:2726] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:00.743017 13158 raft_consensus.cc:729] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.743053 13157 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.745688 13157 ts_tablet_manager.cc:1398] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20240503 08:04:00.746724 13157 tablet_bootstrap.cc:492] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.746870 13181 raft_consensus.cc:504] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.749033 13181 leader_election.cc:288] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.751979 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a484222860f74f7284dbc7fdec58c36f" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:00.752805 12965 raft_consensus.cc:3037] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.751979 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a484222860f74f7284dbc7fdec58c36f" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.753212 13157 tablet_bootstrap.cc:654] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.754032 13155 tablet_bootstrap.cc:492] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.754550 13155 ts_tablet_manager.cc:1367] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.065s user 0.017s sys 0.003s
W20240503 08:04:00.754405 13080 leader_election.cc:341] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.759529 13158 consensus_queue.cc:260] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.760195 12965 raft_consensus.cc:2454] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.761945 13081 leader_election.cc:302] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:00.763352 13181 raft_consensus.cc:2781] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.765595 13158 ts_tablet_manager.cc:1398] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20240503 08:04:00.764139 13155 raft_consensus.cc:348] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.766649 13155 raft_consensus.cc:374] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.766804 13158 tablet_bootstrap.cc:492] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.765079 13181 raft_consensus.cc:686] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:00.767110 13157 tablet_bootstrap.cc:492] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.767555 13155 raft_consensus.cc:729] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.768162 13157 ts_tablet_manager.cc:1367] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.004s
I20240503 08:04:00.768932 13181 consensus_queue.cc:237] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.770758 13155 consensus_queue.cc:260] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.771517 13157 raft_consensus.cc:348] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.772290 13157 raft_consensus.cc:374] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.772596 13157 raft_consensus.cc:729] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.774716 13155 ts_tablet_manager.cc:1398] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.020s user 0.006s sys 0.003s
I20240503 08:04:00.776194 13155 tablet_bootstrap.cc:492] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.782862 13157 consensus_queue.cc:260] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.785022 13157 ts_tablet_manager.cc:1398] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20240503 08:04:00.785806 13157 tablet_bootstrap.cc:492] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.789965 13157 tablet_bootstrap.cc:654] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.782311 13158 tablet_bootstrap.cc:654] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.796059 13157 tablet_bootstrap.cc:492] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.796597 13157 ts_tablet_manager.cc:1367] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.001s
I20240503 08:04:00.798772 13158 tablet_bootstrap.cc:492] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.799263 13158 ts_tablet_manager.cc:1367] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.004s
I20240503 08:04:00.799276 13157 raft_consensus.cc:348] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.800172 13157 raft_consensus.cc:374] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.800463 13157 raft_consensus.cc:729] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.801107 13157 consensus_queue.cc:260] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.801533 13158 raft_consensus.cc:348] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.801051 12864 catalog_manager.cc:5484] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:00.802155 13158 raft_consensus.cc:374] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.802470 13158 raft_consensus.cc:729] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.803048 13158 consensus_queue.cc:260] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.803645 13157 ts_tablet_manager.cc:1398] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20240503 08:04:00.804661 13157 tablet_bootstrap.cc:492] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.805830 13158 ts_tablet_manager.cc:1398] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:00.807087 13158 tablet_bootstrap.cc:492] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.809114 13155 tablet_bootstrap.cc:654] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.810074 13157 tablet_bootstrap.cc:654] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.813251 13158 tablet_bootstrap.cc:654] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.818975 13157 tablet_bootstrap.cc:492] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.819392 13157 ts_tablet_manager.cc:1367] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.005s
I20240503 08:04:00.819855 13158 tablet_bootstrap.cc:492] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.820350 13158 ts_tablet_manager.cc:1367] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20240503 08:04:00.821370 13157 raft_consensus.cc:348] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.822074 13157 raft_consensus.cc:374] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.822329 13157 raft_consensus.cc:729] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.822960 13157 consensus_queue.cc:260] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.823117 12862 catalog_manager.cc:5484] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 1 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:00.825176 13157 ts_tablet_manager.cc:1398] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20240503 08:04:00.825510 13155 tablet_bootstrap.cc:492] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.825950 13155 ts_tablet_manager.cc:1367] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.004s
I20240503 08:04:00.826066 13157 tablet_bootstrap.cc:492] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957: Bootstrap starting.
I20240503 08:04:00.825381 12862 catalog_manager.cc:5484] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 1 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:00.828691 13155 raft_consensus.cc:348] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.829392 13155 raft_consensus.cc:374] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.829667 13155 raft_consensus.cc:729] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.830169 13157 tablet_bootstrap.cc:654] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.830323 13155 consensus_queue.cc:260] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.823078 13158 raft_consensus.cc:348] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.831897 13158 raft_consensus.cc:374] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.832172 13158 raft_consensus.cc:729] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.832613 13155 ts_tablet_manager.cc:1398] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20240503 08:04:00.832955 13158 consensus_queue.cc:260] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.833474 13155 tablet_bootstrap.cc:492] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
W20240503 08:04:00.835707 13067 tablet.cc:2369] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 08:04:00.839028 13158 ts_tablet_manager.cc:1398] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.018s user 0.004s sys 0.004s
I20240503 08:04:00.839783 13155 tablet_bootstrap.cc:654] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.846390 13157 tablet_bootstrap.cc:492] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957: No bootstrap required, opened a new log
I20240503 08:04:00.846870 13157 ts_tablet_manager.cc:1367] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.003s
I20240503 08:04:00.849593 13158 tablet_bootstrap.cc:492] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.849874 13157 raft_consensus.cc:348] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.850716 13157 raft_consensus.cc:374] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.851035 13157 raft_consensus.cc:729] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.851888 13157 consensus_queue.cc:260] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.854315 13157 ts_tablet_manager.cc:1398] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240503 08:04:00.859160 13158 tablet_bootstrap.cc:654] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.865378 13158 tablet_bootstrap.cc:492] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.865732 13155 tablet_bootstrap.cc:492] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.865872 13158 ts_tablet_manager.cc:1367] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.026s user 0.013s sys 0.000s
I20240503 08:04:00.866168 13155 ts_tablet_manager.cc:1367] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.033s user 0.012s sys 0.016s
I20240503 08:04:00.868525 13155 raft_consensus.cc:348] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.868716 13158 raft_consensus.cc:348] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.869240 13155 raft_consensus.cc:374] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.869275 13158 raft_consensus.cc:374] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.869659 13155 raft_consensus.cc:729] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.869719 13158 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.870406 13155 consensus_queue.cc:260] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.870406 13158 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.872591 13155 ts_tablet_manager.cc:1398] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:00.872654 13158 ts_tablet_manager.cc:1398] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20240503 08:04:00.873580 13155 tablet_bootstrap.cc:492] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: Bootstrap starting.
I20240503 08:04:00.873589 13158 tablet_bootstrap.cc:492] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.874339 13182 raft_consensus.cc:482] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.874886 13182 raft_consensus.cc:504] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.877095 13182 leader_election.cc:288] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.878902 13155 tablet_bootstrap.cc:654] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.879725 13158 tablet_bootstrap.cc:654] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.879500 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.880062 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:00.880631 13115 raft_consensus.cc:2454] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 0.
W20240503 08:04:00.880939 12931 leader_election.cc:341] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.881922 12932 leader_election.cc:302] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:00.882558 13182 raft_consensus.cc:2781] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.882901 13182 raft_consensus.cc:482] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.883272 13182 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.885917 13155 tablet_bootstrap.cc:492] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: No bootstrap required, opened a new log
I20240503 08:04:00.886445 13155 ts_tablet_manager.cc:1367] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20240503 08:04:00.889914 13155 raft_consensus.cc:348] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.890394 13182 raft_consensus.cc:504] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.891045 13155 raft_consensus.cc:374] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.891633 13155 raft_consensus.cc:729] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.892640 13155 consensus_queue.cc:260] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.893397 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.894731 13155 ts_tablet_manager.cc:1398] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
W20240503 08:04:00.894690 12931 leader_election.cc:341] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:00.895716 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:00.896200 13115 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.896500 13158 tablet_bootstrap.cc:492] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.896879 13158 ts_tablet_manager.cc:1367] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.002s
I20240503 08:04:00.898563 13158 raft_consensus.cc:348] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.899049 13158 raft_consensus.cc:374] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.899348 13158 raft_consensus.cc:729] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.899264 13182 leader_election.cc:288] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.900175 13158 consensus_queue.cc:260] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.901716 13158 ts_tablet_manager.cc:1398] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 08:04:00.902397 13158 tablet_bootstrap.cc:492] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Bootstrap starting.
I20240503 08:04:00.903019 13115 raft_consensus.cc:2454] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 1.
I20240503 08:04:00.904104 12932 leader_election.cc:302] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:00.905411 13182 raft_consensus.cc:2781] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.905942 13182 raft_consensus.cc:686] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:00.907425 13158 tablet_bootstrap.cc:654] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:00.907400 13182 consensus_queue.cc:237] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.914177 13158 tablet_bootstrap.cc:492] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: No bootstrap required, opened a new log
I20240503 08:04:00.914752 13158 ts_tablet_manager.cc:1367] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.008s
I20240503 08:04:00.917171 13158 raft_consensus.cc:348] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.917670 13158 raft_consensus.cc:374] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:00.917881 13158 raft_consensus.cc:729] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Initialized, Role: FOLLOWER
I20240503 08:04:00.918399 13158 consensus_queue.cc:260] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.918357 12863 catalog_manager.cc:5484] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:00.920465 13158 ts_tablet_manager.cc:1398] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20240503 08:04:00.932570 13181 raft_consensus.cc:482] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.933147 13181 raft_consensus.cc:504] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.935111 13181 leader_election.cc:288] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.936208 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.936388 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.936827 12965 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.937108 13039 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 0.
I20240503 08:04:00.938185 13081 leader_election.cc:302] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:00.938946 13181 raft_consensus.cc:2781] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.939414 13181 raft_consensus.cc:482] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.939689 13181 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.944500 13181 raft_consensus.cc:504] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.946043 13181 leader_election.cc:288] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.946991 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:00.947049 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:00.947688 12965 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.947772 13039 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.953013 13039 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.953006 12965 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 1.
I20240503 08:04:00.954526 13080 leader_election.cc:302] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:00.955564 13181 raft_consensus.cc:2781] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:00.956044 13181 raft_consensus.cc:686] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:00.956699 13181 consensus_queue.cc:237] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.961242 13181 raft_consensus.cc:482] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.961664 13181 raft_consensus.cc:504] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.964119 13181 leader_election.cc:288] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:00.964212 13163 raft_consensus.cc:482] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.965013 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c22dbc72b046788317d845ec3ef7b7" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.965481 12863 catalog_manager.cc:5484] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 1 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:00.966012 13163 raft_consensus.cc:504] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.966781 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c22dbc72b046788317d845ec3ef7b7" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.967677 13039 raft_consensus.cc:2379] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6cfd1f6d1cef47349921f57394a0212c in current term 1: Already voted for candidate 87a73f1536014d609105112116c0e957 in this term.
I20240503 08:04:00.968686 13080 leader_election.cc:302] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c; no voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957
I20240503 08:04:00.969513 13181 raft_consensus.cc:3037] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.976991 13181 raft_consensus.cc:2726] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:00.979295 13163 leader_election.cc:288] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.980834 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:00.981568 12965 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 0.
I20240503 08:04:00.982560 13007 leader_election.cc:302] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:00.983551 13163 raft_consensus.cc:2781] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:00.983949 13163 raft_consensus.cc:482] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:00.984265 13163 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.991565 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:00.991648 13163 raft_consensus.cc:504] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.992488 13115 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 0.
I20240503 08:04:00.992834 13182 raft_consensus.cc:482] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:00.993698 13163 leader_election.cc:288] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.993506 13182 raft_consensus.cc:504] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:00.994943 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:00.995469 12965 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.995640 13182 leader_election.cc:288] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:00.996323 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" is_pre_election: true
I20240503 08:04:00.996806 13114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:00.997200 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:00.997463 13114 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 0.
I20240503 08:04:00.997781 13115 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:00.998720 12932 leader_election.cc:302] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:00.999786 13182 raft_consensus.cc:2781] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:01.000250 13182 raft_consensus.cc:482] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:01.000730 13182 raft_consensus.cc:3037] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.002562 12965 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 1.
I20240503 08:04:01.002856 13115 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 1.
I20240503 08:04:01.003352 13213 raft_consensus.cc:482] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:01.003839 13007 leader_election.cc:302] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.004158 13213 raft_consensus.cc:504] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.004683 13163 raft_consensus.cc:2781] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:01.005195 13039 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 0.
I20240503 08:04:01.005945 13163 raft_consensus.cc:686] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:01.007705 13213 leader_election.cc:288] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.007726 13182 raft_consensus.cc:504] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.007684 13163 consensus_queue.cc:237] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.008857 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:01.009372 13115 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 0.
I20240503 08:04:01.009907 13182 leader_election.cc:288] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.011204 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:01.011530 13007 leader_election.cc:302] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:01.011049 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.012317 12965 raft_consensus.cc:2379] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in current term 1: Already voted for candidate 87a73f1536014d609105112116c0e957 in this term.
I20240503 08:04:01.012516 13115 raft_consensus.cc:3037] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.012612 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:01.013334 13039 raft_consensus.cc:3037] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.018875 13039 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 1.
I20240503 08:04:01.019445 13213 raft_consensus.cc:2741] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20240503 08:04:01.020025 12931 leader_election.cc:302] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.020823 13182 raft_consensus.cc:2781] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:01.020977 13115 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 1.
I20240503 08:04:01.021234 13182 raft_consensus.cc:686] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:01.022116 13182 consensus_queue.cc:237] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.025609 12863 catalog_manager.cc:5484] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.031942 12862 catalog_manager.cc:5484] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.054494 13169 consensus_queue.cc:1035] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.057049 13169 consensus_queue.cc:1035] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.110934 13163 raft_consensus.cc:482] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:01.111460 13163 raft_consensus.cc:504] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.113122 13163 leader_election.cc:288] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.114183 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a484222860f74f7284dbc7fdec58c36f" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:01.114403 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a484222860f74f7284dbc7fdec58c36f" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:01.114917 12965 raft_consensus.cc:2379] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in current term 1: Already voted for candidate 6cfd1f6d1cef47349921f57394a0212c in this term.
I20240503 08:04:01.116256 13007 leader_election.cc:302] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9; no voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957
I20240503 08:04:01.116910 13163 raft_consensus.cc:3037] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.121850 13163 raft_consensus.cc:2726] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:01.128376 13181 consensus_queue.cc:1035] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.139533 13181 consensus_queue.cc:1035] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:01.155007 13163 raft_consensus.cc:482] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:01.155625 13163 raft_consensus.cc:504] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.157524 13163 leader_election.cc:288] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.158511 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:01.158655 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:01.159477 13115 raft_consensus.cc:2379] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in current term 1: Already voted for candidate 87a73f1536014d609105112116c0e957 in this term.
I20240503 08:04:01.160591 13007 leader_election.cc:302] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9; no voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957
I20240503 08:04:01.161499 13163 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.162232 13164 consensus_queue.cc:1035] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.169040 13163 raft_consensus.cc:2726] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:01.178458 13164 consensus_queue.cc:1035] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.184284 13163 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:01.184896 13163 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.186784 13163 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.187549 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:01.188361 12965 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 0.
I20240503 08:04:01.189514 13007 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.190055 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:01.190568 13163 raft_consensus.cc:2781] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:01.190726 13115 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 0.
I20240503 08:04:01.191046 13163 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:01.191460 13163 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.196761 13163 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.198269 13163 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.199047 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:01.199286 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.199594 12965 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.199769 13115 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.204541 12965 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 1.
I20240503 08:04:01.204560 13115 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 1.
I20240503 08:04:01.206034 13007 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.207020 13163 raft_consensus.cc:2781] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:01.207687 13163 raft_consensus.cc:686] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:01.208633 13181 consensus_queue.cc:1035] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.208647 13163 consensus_queue.cc:237] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.216924 12862 catalog_manager.cc:5484] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.219806 13181 consensus_queue.cc:1035] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.274247 13163 raft_consensus.cc:482] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:01.274694 13163 raft_consensus.cc:504] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.276233 13163 leader_election.cc:288] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.277109 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a73f1536014d609105112116c0e957" is_pre_election: true
I20240503 08:04:01.277227 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cfd1f6d1cef47349921f57394a0212c" is_pre_election: true
I20240503 08:04:01.277896 13115 raft_consensus.cc:2379] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in current term 1: Already voted for candidate 87a73f1536014d609105112116c0e957 in this term.
I20240503 08:04:01.278685 13007 leader_election.cc:302] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9; no voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957
I20240503 08:04:01.279289 13163 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:01.283877 13163 raft_consensus.cc:2726] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:01.349813 13181 consensus_queue.cc:1035] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.350636 13159 consensus_queue.cc:1035] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.364233 13231 consensus_queue.cc:1035] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:01.366295 13231 consensus_queue.cc:1035] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.401870 13163 consensus_queue.cc:1035] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.412176 13163 consensus_queue.cc:1035] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.429379 13164 consensus_queue.cc:1035] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.439893 13164 consensus_queue.cc:1035] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.469522 13115 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:51514
I20240503 08:04:01.470108 13115 raft_consensus.cc:595] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:01.475656 13115 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:51514
I20240503 08:04:01.476310 13115 raft_consensus.cc:595] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:01.496925 13039 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:56392
I20240503 08:04:01.497558 13039 raft_consensus.cc:595] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:01.498448 13164 consensus_queue.cc:1035] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:01.510375 13169 consensus_queue.cc:1035] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.521862 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5c013c695821420597e5385fd5a679a1"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:59594
I20240503 08:04:01.522374 12965 raft_consensus.cc:595] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Received request to transfer leadership to TS 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:01.527412 13115 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
mode: GRACEFUL
new_leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:51514
I20240503 08:04:01.528096 13115 raft_consensus.cc:595] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Received request to transfer leadership to TS 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:01.533133 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:59594
I20240503 08:04:01.533675 12965 raft_consensus.cc:595] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Received request to transfer leadership to TS 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:01.595597 13163 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:01.607014 13163 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:01.616808 13163 raft_consensus.cc:979] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:01.617213 13213 raft_consensus.cc:1067] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:01.618580 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59586
I20240503 08:04:01.619078 12965 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:01.619432 12965 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.624226 12965 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.625914 12965 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.626889 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:01.627118 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.627522 13039 raft_consensus.cc:3032] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:01.627804 13115 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.627853 13039 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:01.628767 13039 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.630113 13039 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.634212 13115 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:01.635777 12932 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.636334 13039 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:01.636544 13169 raft_consensus.cc:2781] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:01.637014 13181 raft_consensus.cc:979] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:01.637087 13169 raft_consensus.cc:686] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:01.637516 13181 raft_consensus.cc:1067] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:01.638043 13169 consensus_queue.cc:237] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.639070 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59584
I20240503 08:04:01.639750 12965 raft_consensus.cc:482] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:01.640122 12965 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.645761 12965 raft_consensus.cc:504] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:01.645859 12862 catalog_manager.cc:5484] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 1 to 2, leader changed from 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.647847 12965 leader_election.cc:288] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Requested vote from peers 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:01.648721 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.648999 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:01.649365 13115 raft_consensus.cc:3032] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:01.649632 13039 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.649744 13115 raft_consensus.cc:729] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:01.650610 13115 consensus_queue.cc:260] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:01.651788 13115 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.655568 13039 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:01.656611 12931 leader_election.cc:302] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.657341 13169 raft_consensus.cc:2781] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:01.657712 13169 raft_consensus.cc:686] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:01.658444 13115 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:01.658437 13169 consensus_queue.cc:237] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:01.665261 12862 catalog_manager.cc:5484] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 1 to 2, leader changed from 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.700850 13181 raft_consensus.cc:979] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:01.701265 13183 raft_consensus.cc:1067] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:01.702665 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59584
I20240503 08:04:01.703186 12965 raft_consensus.cc:482] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:01.703469 12965 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.708003 12965 raft_consensus.cc:504] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.709466 12965 leader_election.cc:288] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.710342 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:01.710491 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.710870 13039 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.710965 13115 raft_consensus.cc:3032] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:01.711293 13115 raft_consensus.cc:729] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:01.711923 13115 consensus_queue.cc:260] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.712837 13115 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.716859 13039 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:01.717180 13115 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:01.717999 12931 leader_election.cc:302] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.718672 13169 raft_consensus.cc:2781] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:01.719120 13169 raft_consensus.cc:686] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:01.719899 13169 consensus_queue.cc:237] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.725596 12862 catalog_manager.cc:5484] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 1 to 2, leader changed from 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.888360 13169 raft_consensus.cc:979] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: : Instructing follower 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:01.888787 13164 raft_consensus.cc:1067] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Signalling peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:01.890374 13039 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56372
I20240503 08:04:01.890858 13039 raft_consensus.cc:482] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:01.891201 13039 raft_consensus.cc:3037] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.896966 13039 raft_consensus.cc:504] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.898466 13039 leader_election.cc:288] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 2 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.900055 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:01.900682 12965 raft_consensus.cc:3032] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:01.900837 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa48fa501614e9d88dababf62e20b36" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.901046 12965 raft_consensus.cc:729] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:01.901440 13115 raft_consensus.cc:3037] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.901826 12965 consensus_queue.cc:260] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.903163 12965 raft_consensus.cc:3037] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.908403 13115 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 2.
I20240503 08:04:01.909765 13007 leader_election.cc:302] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:01.909873 12965 raft_consensus.cc:2454] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 2.
I20240503 08:04:01.910630 13213 raft_consensus.cc:2781] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:01.911078 13213 raft_consensus.cc:686] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:01.911922 13213 consensus_queue.cc:237] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.917724 12862 catalog_manager.cc:5484] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 1 to 2, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 2 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.925869 13169 raft_consensus.cc:979] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: : Instructing follower 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:01.926395 13164 raft_consensus.cc:1067] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Signalling peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:01.928030 13039 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5c013c695821420597e5385fd5a679a1"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56372
I20240503 08:04:01.928635 13039 raft_consensus.cc:482] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:01.928997 13039 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.933805 13183 raft_consensus.cc:979] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: : Instructing follower 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:01.935274 13181 raft_consensus.cc:1067] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Signalling peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:01.936403 13039 raft_consensus.cc:504] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.938493 13039 leader_election.cc:288] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 2 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.939517 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:01.940193 12965 raft_consensus.cc:3032] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:01.940609 12965 raft_consensus.cc:729] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:01.940734 13039 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56388
I20240503 08:04:01.941188 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.941531 13039 raft_consensus.cc:482] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:01.942010 13115 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.942075 13039 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.941790 12965 consensus_queue.cc:260] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.943094 12965 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.946801 13115 raft_consensus.cc:2454] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 2.
I20240503 08:04:01.947901 13007 leader_election.cc:302] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:01.948583 13214 raft_consensus.cc:2781] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:01.948963 13214 raft_consensus.cc:686] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:01.949214 13039 raft_consensus.cc:504] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.949957 12965 raft_consensus.cc:2454] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 2.
I20240503 08:04:01.949815 13214 consensus_queue.cc:237] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.951237 13039 leader_election.cc:288] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 2 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:01.952296 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:01.952415 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:01.953073 13115 raft_consensus.cc:3032] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:01.953085 12965 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.953547 13115 raft_consensus.cc:729] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:01.954334 13115 consensus_queue.cc:260] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.955600 13115 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:01.957378 12862 catalog_manager.cc:5484] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 1 to 2, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 2 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:01.960112 12965 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 2.
I20240503 08:04:01.961503 13007 leader_election.cc:302] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:01.962388 13115 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 2.
I20240503 08:04:01.963928 13214 raft_consensus.cc:2781] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:01.964323 13214 raft_consensus.cc:686] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:01.965068 13214 consensus_queue.cc:237] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:01.974951 12863 catalog_manager.cc:5484] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 1 to 2, leader changed from 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3) to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 2 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:02.043550 13115 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.044749 13169 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.055967 13039 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.057610 13247 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.099326 13115 raft_consensus.cc:1261] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.100893 13247 consensus_queue.cc:1035] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.101512 13039 raft_consensus.cc:1261] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.102742 13247 consensus_queue.cc:1035] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.154026 13115 raft_consensus.cc:1261] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.155334 13247 consensus_queue.cc:1035] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:02.164367 13039 raft_consensus.cc:1261] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.166167 13169 consensus_queue.cc:1035] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.349386 12965 raft_consensus.cc:1261] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.350626 13214 consensus_queue.cc:1035] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:02.358431 13115 raft_consensus.cc:1261] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.360239 13213 consensus_queue.cc:1035] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.385468 12965 raft_consensus.cc:1261] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.386919 13163 consensus_queue.cc:1035] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:02.395020 13115 raft_consensus.cc:1261] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.397765 13214 consensus_queue.cc:1035] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:02.431283 12965 raft_consensus.cc:1261] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.432896 13214 consensus_queue.cc:1035] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:02.442083 13115 raft_consensus.cc:1261] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:02.443387 13163 consensus_queue.cc:1035] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:04.564378 12312 auto_leader_rebalancer-test.cc:331] The leader distribution is
I20240503 08:04:04.565121 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:04.587317 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:56922
I20240503 08:04:04.587800 12965 raft_consensus.cc:595] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Received request to transfer leadership to TS 6cfd1f6d1cef47349921f57394a0212c
I20240503 08:04:04.589787 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c088bd75653841da80100a14604827b9"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:56922
I20240503 08:04:04.590198 12965 raft_consensus.cc:595] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Received request to transfer leadership to TS 6cfd1f6d1cef47349921f57394a0212c
I20240503 08:04:04.591043 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240503 08:04:04.591500 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:04.633988 13251 raft_consensus.cc:979] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: : Instructing follower 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:04.634416 13251 raft_consensus.cc:1067] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Signalling peer 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:04.635768 13114 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:51500
I20240503 08:04:04.636225 13114 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:04.636498 13114 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:04.641276 13114 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:04.642786 13114 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 3 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:04.643759 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:04.643769 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:04.644450 12965 raft_consensus.cc:3032] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:04.644493 13041 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:04.644876 12965 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:04.645721 12965 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:04.646934 12965 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:04.651618 13041 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 3.
I20240503 08:04:04.652918 13080 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:04.653036 12965 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 3.
I20240503 08:04:04.653649 13252 raft_consensus.cc:2781] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:04.654774 13252 raft_consensus.cc:686] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:04.655619 13252 consensus_queue.cc:237] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:04.666709 12862 catalog_manager.cc:5484] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 2 to 3, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 3 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:04.959782 13253 raft_consensus.cc:979] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: : Instructing follower 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:04.960218 13253 raft_consensus.cc:1067] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Signalling peer 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:04.961520 13115 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c088bd75653841da80100a14604827b9"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:51500
I20240503 08:04:04.961966 13115 raft_consensus.cc:482] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:04.962194 13115 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:04.967751 13115 raft_consensus.cc:504] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:04.969417 13115 leader_election.cc:288] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 2 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:04.970017 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:04.970593 12965 raft_consensus.cc:3032] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:04.970839 12965 raft_consensus.cc:729] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:04.970808 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:04.971598 13041 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:04.971824 12965 consensus_queue.cc:260] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:04.972820 12965 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:04.977422 13041 raft_consensus.cc:2454] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 2.
I20240503 08:04:04.978018 12965 raft_consensus.cc:2454] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 2.
I20240503 08:04:04.978485 13080 leader_election.cc:302] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:04.979390 13252 raft_consensus.cc:2781] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:04.979806 13252 raft_consensus.cc:686] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:04.980427 13252 consensus_queue.cc:237] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:04.989495 12862 catalog_manager.cc:5484] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 1 to 2, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 2 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:05.220151 12965 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:05.221262 13252 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:05.229780 13041 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:05.231426 13252 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:05.502318 12965 raft_consensus.cc:1261] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:05.503582 13252 consensus_queue.cc:1035] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:05.512660 13041 raft_consensus.cc:1261] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:05.513952 13256 consensus_queue.cc:1035] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:05.592394 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:05.597636 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:05.598028 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:06.598843 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:06.605180 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:06.605618 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:07.606484 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:07.611888 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:07.612308 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:08.613255 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:08.620539 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:08.620911 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:09.621676 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:09.626649 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:09.627065 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:10.627864 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:10.633188 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:10.633598 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:11.634449 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:11.639868 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:11.640267 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:12.641013 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:12.645973 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:12.646350 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:13.647226 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:13.653069 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:13.653462 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:14.678364 12312 auto_leader_rebalancer-test.cc:350] The leader distribution is
I20240503 08:04:14.687952 13115 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c088bd75653841da80100a14604827b9"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:51514
I20240503 08:04:14.688592 13115 raft_consensus.cc:595] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:14.696094 13115 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:51514
I20240503 08:04:14.696571 13115 raft_consensus.cc:595] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:14.701159 13041 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:56392
I20240503 08:04:14.701635 13041 raft_consensus.cc:595] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:14.705999 13041 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5c013c695821420597e5385fd5a679a1"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:56392
I20240503 08:04:14.706424 13041 raft_consensus.cc:595] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:14.710301 13041 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
mode: GRACEFUL
new_leader_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:56392
I20240503 08:04:14.710728 13041 raft_consensus.cc:595] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Received request to transfer leadership to TS 87a73f1536014d609105112116c0e957
I20240503 08:04:14.838140 13272 raft_consensus.cc:979] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:14.838619 13272 raft_consensus.cc:1067] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:14.840090 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5c013c695821420597e5385fd5a679a1"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59586
I20240503 08:04:14.840610 12965 raft_consensus.cc:482] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:14.841077 12965 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:14.846186 12965 raft_consensus.cc:504] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:14.848114 12965 leader_election.cc:288] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 3 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:14.849020 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:14.848994 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c013c695821420597e5385fd5a679a1" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:14.849956 13115 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:14.850085 13041 raft_consensus.cc:3032] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:14.850466 13041 raft_consensus.cc:729] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:14.851337 13041 consensus_queue.cc:260] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:14.852622 13041 raft_consensus.cc:3037] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:14.857007 13115 raft_consensus.cc:2454] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 3.
I20240503 08:04:14.858273 12932 leader_election.cc:302] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:14.859033 13041 raft_consensus.cc:2454] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 3.
I20240503 08:04:14.859117 13268 raft_consensus.cc:2781] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:14.860324 13268 raft_consensus.cc:686] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 3 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:14.861119 13268 consensus_queue.cc:237] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:14.867691 12862 catalog_manager.cc:5484] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 2 to 3, leader changed from 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 3 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:14.943976 13274 raft_consensus.cc:979] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:14.944401 13274 raft_consensus.cc:1067] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:14.945834 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c088bd75653841da80100a14604827b9"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59584
I20240503 08:04:14.946229 12965 raft_consensus.cc:482] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:14.946506 12965 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:14.952185 12965 raft_consensus.cc:504] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:14.953506 12965 leader_election.cc:288] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 3 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:14.954239 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:14.954402 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c088bd75653841da80100a14604827b9" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:14.954700 13041 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:14.954874 13115 raft_consensus.cc:3032] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:14.955078 13115 raft_consensus.cc:729] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:14.955721 13115 consensus_queue.cc:260] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:14.956637 13115 raft_consensus.cc:3037] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:14.958577 13041 raft_consensus.cc:2454] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 3.
I20240503 08:04:14.959614 12931 leader_election.cc:302] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:14.960227 13268 raft_consensus.cc:2781] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:14.960755 13268 raft_consensus.cc:686] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 3 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:14.960892 13115 raft_consensus.cc:2454] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 3.
I20240503 08:04:14.961773 13268 consensus_queue.cc:237] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:14.968780 12862 catalog_manager.cc:5484] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 2 to 3, leader changed from 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 3 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:15.054402 13278 raft_consensus.cc:979] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:15.054827 13278 raft_consensus.cc:1067] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:15.056119 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59586
I20240503 08:04:15.056579 12965 raft_consensus.cc:482] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:15.056807 12965 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:15.060801 12965 raft_consensus.cc:504] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.062080 12965 leader_election.cc:288] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:15.062958 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:15.063024 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:15.063608 13041 raft_consensus.cc:3032] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:15.063612 13115 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:15.064100 13041 raft_consensus.cc:729] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:15.064697 13041 consensus_queue.cc:260] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.065816 13041 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:15.068568 13115 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:15.069540 12932 leader_election.cc:302] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:15.070125 13041 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 2.
I20240503 08:04:15.070219 13268 raft_consensus.cc:2781] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:15.070660 13268 raft_consensus.cc:686] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:15.071573 13268 consensus_queue.cc:237] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.077934 12862 catalog_manager.cc:5484] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 1 to 2, leader changed from 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:15.118391 13272 raft_consensus.cc:979] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:15.118865 13272 raft_consensus.cc:1067] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:15.120160 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59586
I20240503 08:04:15.120724 12965 raft_consensus.cc:482] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:15.121060 12965 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:15.125499 12965 raft_consensus.cc:504] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.126926 12965 leader_election.cc:288] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 3 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:15.127887 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:15.128053 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e6c2a47f6243ae9d98439b9cb5ff52" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:15.128592 13115 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:15.128553 13041 raft_consensus.cc:3032] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:15.129158 13041 raft_consensus.cc:729] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:15.130064 13041 consensus_queue.cc:260] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.131314 13041 raft_consensus.cc:3037] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:15.133553 13115 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 3.
I20240503 08:04:15.134507 12932 leader_election.cc:302] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:15.135454 13279 raft_consensus.cc:2781] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:15.135879 13279 raft_consensus.cc:686] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 3 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:15.136515 13041 raft_consensus.cc:2454] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 3.
I20240503 08:04:15.136716 13279 consensus_queue.cc:237] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.143419 12862 catalog_manager.cc:5484] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 2 to 3, leader changed from 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 3 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:15.150418 13277 raft_consensus.cc:979] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: : Instructing follower 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:15.150946 13277 raft_consensus.cc:1067] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Signalling peer 87a73f1536014d609105112116c0e957 to start an election
I20240503 08:04:15.152325 12965 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "87a73f1536014d609105112116c0e957"
from {username='slave'} at 127.0.0.1:59584
I20240503 08:04:15.152742 12965 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:15.152977 12965 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:04:15.156952 12965 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.158278 12965 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 4 election: Requested vote from peers 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:15.159102 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:15.159307 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "87a73f1536014d609105112116c0e957" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:15.159685 13041 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:04:15.159770 13115 raft_consensus.cc:3032] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Stepping down as leader of term 3
I20240503 08:04:15.160115 13115 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:15.160867 13115 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.162051 13115 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Advancing to term 4
I20240503 08:04:15.165936 13041 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 4.
I20240503 08:04:15.166886 12931 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:15.167536 13279 raft_consensus.cc:2781] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 08:04:15.167840 13115 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a73f1536014d609105112116c0e957 in term 4.
I20240503 08:04:15.168037 13279 raft_consensus.cc:686] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 LEADER]: Becoming Leader. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:15.168922 13279 consensus_queue.cc:237] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:15.175796 12862 catalog_manager.cc:5484] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 reported cstate change: term changed from 3 to 4, leader changed from 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3) to 87a73f1536014d609105112116c0e957 (127.12.6.1). New cstate: current_term: 4 leader_uuid: "87a73f1536014d609105112116c0e957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:15.341847 13115 raft_consensus.cc:1261] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:15.343065 13279 consensus_queue.cc:1035] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:15.354503 13041 raft_consensus.cc:1261] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:15.355623 13279 consensus_queue.cc:1035] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:15.467347 13041 raft_consensus.cc:1261] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:15.468405 13279 consensus_queue.cc:1035] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:15.477059 13115 raft_consensus.cc:1261] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:15.478526 13279 consensus_queue.cc:1035] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:15.501901 13041 raft_consensus.cc:1261] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:15.503120 13295 consensus_queue.cc:1035] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:15.513394 13115 raft_consensus.cc:1261] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:15.514640 13295 consensus_queue.cc:1035] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:15.601608 13041 raft_consensus.cc:1261] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:15.602761 13295 consensus_queue.cc:1035] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:15.612452 13115 raft_consensus.cc:1261] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:15.614895 13279 consensus_queue.cc:1035] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:15.639935 13115 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 4 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:04:15.641003 13279 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:15.648816 13041 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 4 FOLLOWER]: Refusing update from remote peer 87a73f1536014d609105112116c0e957: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 08:04:15.650753 13268 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:17.740763 12312 auto_leader_rebalancer-test.cc:365] The leader distribution is
I20240503 08:04:17.741451 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:17.749212 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56922
I20240503 08:04:17.749835 12965 raft_consensus.cc:595] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Received request to transfer leadership to TS 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:17.751996 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56922
I20240503 08:04:17.752437 12965 raft_consensus.cc:595] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 LEADER]: Received request to transfer leadership to TS 4172ea1e5a3e4cd2b3aa51f3cee814c9
I20240503 08:04:17.754338 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:56922
I20240503 08:04:17.754750 12965 raft_consensus.cc:595] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Received request to transfer leadership to TS 6cfd1f6d1cef47349921f57394a0212c
I20240503 08:04:17.756664 12965 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971"
dest_uuid: "87a73f1536014d609105112116c0e957"
mode: GRACEFUL
new_leader_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:56922
I20240503 08:04:17.757081 12965 raft_consensus.cc:595] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Received request to transfer leadership to TS 6cfd1f6d1cef47349921f57394a0212c
I20240503 08:04:17.757933 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 4
I20240503 08:04:17.758337 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:18.090840 13305 raft_consensus.cc:979] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: : Instructing follower 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:18.091219 13295 raft_consensus.cc:979] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: : Instructing follower 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:18.092294 13304 raft_consensus.cc:1067] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Signalling peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:18.092159 13305 raft_consensus.cc:1067] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Signalling peer 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:18.094861 13115 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:51500
I20240503 08:04:18.095301 13041 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56372
I20240503 08:04:18.095613 13115 raft_consensus.cc:482] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:18.095795 13041 raft_consensus.cc:482] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:18.096097 13115 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.096203 13041 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.102404 13115 raft_consensus.cc:504] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.102813 13041 raft_consensus.cc:504] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.104996 13041 leader_election.cc:288] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 3 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:18.105340 13115 leader_election.cc:288] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 3 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:18.105981 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:18.106456 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:18.107293 13114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f33b65ff9684d38a7671f1f88e0c190" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:18.107476 13041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916c02f4a324485d883ead3fdd81f7e1" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:18.107481 12963 raft_consensus.cc:3032] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:18.107347 12965 raft_consensus.cc:3032] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:18.108316 13114 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.108306 12963 raft_consensus.cc:729] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:18.108407 12965 raft_consensus.cc:729] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:18.108400 13041 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.109611 12965 consensus_queue.cc:260] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.109591 12963 consensus_queue.cc:260] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.111531 12965 raft_consensus.cc:3037] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.111776 12963 raft_consensus.cc:3037] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.115890 13041 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 3.
I20240503 08:04:18.116741 13114 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 3.
I20240503 08:04:18.117777 13080 leader_election.cc:302] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:18.119601 13007 leader_election.cc:302] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:18.121634 12963 raft_consensus.cc:2454] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 3.
I20240503 08:04:18.121784 13272 raft_consensus.cc:2781] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:18.122458 12965 raft_consensus.cc:2454] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 3.
I20240503 08:04:18.124017 13277 raft_consensus.cc:2781] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:18.125435 13277 raft_consensus.cc:686] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:18.126493 13272 raft_consensus.cc:686] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:18.126724 13277 consensus_queue.cc:237] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.127384 13272 consensus_queue.cc:237] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.137745 12861 catalog_manager.cc:5484] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 2 to 3, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 3 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:18.137748 12862 catalog_manager.cc:5484] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 2 to 3, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 3 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:18.226809 13306 raft_consensus.cc:979] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: : Instructing follower 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:18.227324 13304 raft_consensus.cc:1067] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Signalling peer 6cfd1f6d1cef47349921f57394a0212c to start an election
I20240503 08:04:18.228641 13115 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971"
dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
from {username='slave'} at 127.0.0.1:51500
I20240503 08:04:18.229103 13115 raft_consensus.cc:482] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:18.229439 13115 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.231892 13306 raft_consensus.cc:979] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: : Instructing follower 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:18.232390 13304 raft_consensus.cc:1067] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 LEADER]: Signalling peer 4172ea1e5a3e4cd2b3aa51f3cee814c9 to start an election
I20240503 08:04:18.233668 13041 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858"
dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
from {username='slave'} at 127.0.0.1:56372
I20240503 08:04:18.234066 13041 raft_consensus.cc:482] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 08:04:18.234383 13041 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 4 FOLLOWER]: Advancing to term 5
I20240503 08:04:18.234737 13115 raft_consensus.cc:504] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:18.236258 13115 leader_election.cc:288] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 3 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2:33877)
I20240503 08:04:18.237004 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:18.237551 12965 raft_consensus.cc:3032] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 08:04:18.237488 13039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc22dbb28441cb642e218aa692971" candidate_uuid: "6cfd1f6d1cef47349921f57394a0212c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9"
I20240503 08:04:18.237900 12965 raft_consensus.cc:729] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:18.238131 13039 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.238596 12965 consensus_queue.cc:260] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:18.239521 12965 raft_consensus.cc:3037] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Advancing to term 3
I20240503 08:04:18.240286 13041 raft_consensus.cc:504] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.242725 12963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "87a73f1536014d609105112116c0e957"
I20240503 08:04:18.242919 13115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7396a6e047f14194afdfaa32186a8858" candidate_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6cfd1f6d1cef47349921f57394a0212c"
I20240503 08:04:18.243311 12963 raft_consensus.cc:3032] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 LEADER]: Stepping down as leader of term 4
I20240503 08:04:18.243484 13115 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 4 FOLLOWER]: Advancing to term 5
I20240503 08:04:18.243683 12963 raft_consensus.cc:729] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 87a73f1536014d609105112116c0e957, State: Running, Role: LEADER
I20240503 08:04:18.244544 12965 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 3.
I20240503 08:04:18.244617 13039 raft_consensus.cc:2454] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cfd1f6d1cef47349921f57394a0212c in term 3.
I20240503 08:04:18.244596 12963 consensus_queue.cc:260] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.246012 12963 raft_consensus.cc:3037] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 4 FOLLOWER]: Advancing to term 5
I20240503 08:04:18.245920 13081 leader_election.cc:302] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cfd1f6d1cef47349921f57394a0212c, 87a73f1536014d609105112116c0e957; no voters:
I20240503 08:04:18.246906 13277 raft_consensus.cc:2781] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Leader election won for term 3
I20240503 08:04:18.247750 13277 raft_consensus.cc:686] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Becoming Leader. State: Replica: 6cfd1f6d1cef47349921f57394a0212c, State: Running, Role: LEADER
I20240503 08:04:18.249230 13115 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 5.
I20240503 08:04:18.248754 13277 consensus_queue.cc:237] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } }
I20240503 08:04:18.250267 13007 leader_election.cc:302] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4172ea1e5a3e4cd2b3aa51f3cee814c9, 6cfd1f6d1cef47349921f57394a0212c; no voters:
I20240503 08:04:18.250878 13041 leader_election.cc:288] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [CANDIDATE]: Term 5 election: Requested vote from peers 87a73f1536014d609105112116c0e957 (127.12.6.1:34959), 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3:33615)
I20240503 08:04:18.250910 13272 raft_consensus.cc:2781] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 5 FOLLOWER]: Leader election won for term 5
I20240503 08:04:18.251603 13272 raft_consensus.cc:686] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 5 LEADER]: Becoming Leader. State: Replica: 4172ea1e5a3e4cd2b3aa51f3cee814c9, State: Running, Role: LEADER
I20240503 08:04:18.252372 13272 consensus_queue.cc:237] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } }
I20240503 08:04:18.253717 12963 raft_consensus.cc:2454] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4172ea1e5a3e4cd2b3aa51f3cee814c9 in term 5.
I20240503 08:04:18.260661 12861 catalog_manager.cc:5484] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 reported cstate change: term changed from 4 to 5, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 4172ea1e5a3e4cd2b3aa51f3cee814c9 (127.12.6.2). New cstate: current_term: 5 leader_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:18.260779 12862 catalog_manager.cc:5484] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c reported cstate change: term changed from 2 to 3, leader changed from 87a73f1536014d609105112116c0e957 (127.12.6.1) to 6cfd1f6d1cef47349921f57394a0212c (127.12.6.3). New cstate: current_term: 3 leader_uuid: "6cfd1f6d1cef47349921f57394a0212c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:18.551713 12963 raft_consensus.cc:1261] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:18.552850 13272 consensus_queue.cc:1035] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:18.564889 13115 raft_consensus.cc:1261] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:18.564947 13041 raft_consensus.cc:1261] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:18.566692 13307 consensus_queue.cc:1035] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:18.567248 13277 consensus_queue.cc:1035] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:18.585997 12963 raft_consensus.cc:1261] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:18.587477 13277 consensus_queue.cc:1035] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:18.646359 12963 raft_consensus.cc:1261] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:18.647418 13277 consensus_queue.cc:1035] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:18.657253 13041 raft_consensus.cc:1261] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Refusing update from remote peer 6cfd1f6d1cef47349921f57394a0212c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 08:04:18.658618 13327 consensus_queue.cc:1035] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4172ea1e5a3e4cd2b3aa51f3cee814c9" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:18.694245 13115 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 5 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240503 08:04:18.695437 13321 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cfd1f6d1cef47349921f57394a0212c" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240503 08:04:18.703919 12963 raft_consensus.cc:1261] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 5 FOLLOWER]: Refusing update from remote peer 4172ea1e5a3e4cd2b3aa51f3cee814c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240503 08:04:18.705796 13272 consensus_queue.cc:1035] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a73f1536014d609105112116c0e957" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240503 08:04:18.759403 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:18.766587 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:18.767017 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:19.767933 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:19.772992 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:19.773402 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:20.774210 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:20.780534 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:20.780956 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:21.781666 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:21.788058 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:21.788439 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:22.789211 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:22.794395 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:22.794828 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:23.795648 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:23.801155 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:23.801623 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:24.802443 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:24.807600 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:24.808060 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:25.808876 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:25.814366 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:25.814836 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:26.815685 12312 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 08:04:26.821050 12312 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 08:04:26.821508 12312 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 08:04:27.846060 12312 auto_leader_rebalancer-test.cc:376] The leader distribution is
I20240503 08:04:27.846576 12312 tablet_server.cc:178] TabletServer@127.12.6.1:0 shutting down...
I20240503 08:04:27.874997 12312 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 08:04:27.875792 12312 tablet_replica.cc:330] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.876381 12312 raft_consensus.cc:2229] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.877164 12312 raft_consensus.cc:2258] T 5c013c695821420597e5385fd5a679a1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.878947 12312 tablet_replica.cc:330] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.879521 12312 raft_consensus.cc:2229] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.879906 12312 raft_consensus.cc:2258] T 9aa48fa501614e9d88dababf62e20b36 P 87a73f1536014d609105112116c0e957 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.881543 12312 tablet_replica.cc:330] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.882016 12312 raft_consensus.cc:2229] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.882375 12312 raft_consensus.cc:2258] T 3f33b65ff9684d38a7671f1f88e0c190 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.883931 12312 tablet_replica.cc:330] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.884433 12312 raft_consensus.cc:2229] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.884799 12312 raft_consensus.cc:2258] T a484222860f74f7284dbc7fdec58c36f P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.886340 12312 tablet_replica.cc:330] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.886809 12312 raft_consensus.cc:2229] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 5 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.887233 12312 raft_consensus.cc:2258] T 7396a6e047f14194afdfaa32186a8858 P 87a73f1536014d609105112116c0e957 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.888711 12312 tablet_replica.cc:330] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.889180 12312 raft_consensus.cc:2229] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.889885 12312 raft_consensus.cc:2258] T c088bd75653841da80100a14604827b9 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.891531 12312 tablet_replica.cc:330] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.891989 12312 raft_consensus.cc:2229] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.892347 12312 raft_consensus.cc:2258] T 916c02f4a324485d883ead3fdd81f7e1 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.893857 12312 tablet_replica.cc:330] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.894315 12312 raft_consensus.cc:2229] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.894685 12312 raft_consensus.cc:2258] T b05bc22dbb28441cb642e218aa692971 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.896181 12312 tablet_replica.cc:330] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.896622 12312 raft_consensus.cc:2229] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.897331 12312 raft_consensus.cc:2258] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 87a73f1536014d609105112116c0e957 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.898849 12312 tablet_replica.cc:330] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957: stopping tablet replica
I20240503 08:04:27.899335 12312 raft_consensus.cc:2229] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 1 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.900038 12312 raft_consensus.cc:2258] T 05c22dbc72b046788317d845ec3ef7b7 P 87a73f1536014d609105112116c0e957 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 08:04:27.918288 13081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.6.1:34959: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20240503 08:04:27.919056 12312 tablet_server.cc:194] TabletServer@127.12.6.1:0 shutdown complete.
W20240503 08:04:27.921401 13081 consensus_peers.cc:485] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c -> Peer 87a73f1536014d609105112116c0e957 (127.12.6.1:34959): Couldn't send request to peer 87a73f1536014d609105112116c0e957. Status: Network error: Client connection negotiation failed: client connection to 127.12.6.1:34959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240503 08:04:27.937536 13081 consensus_peers.cc:485] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c -> Peer 87a73f1536014d609105112116c0e957 (127.12.6.1:34959): Couldn't send request to peer 87a73f1536014d609105112116c0e957. Status: Network error: Client connection negotiation failed: client connection to 127.12.6.1:34959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:27.938694 12312 tablet_server.cc:178] TabletServer@127.12.6.2:0 shutting down...
I20240503 08:04:27.965977 12312 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 08:04:27.966703 12312 tablet_replica.cc:330] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.967717 12312 raft_consensus.cc:2229] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.968387 12312 raft_consensus.cc:2258] T 5c013c695821420597e5385fd5a679a1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.970126 12312 tablet_replica.cc:330] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.970620 12312 raft_consensus.cc:2229] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.971331 12312 raft_consensus.cc:2258] T 9aa48fa501614e9d88dababf62e20b36 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.972999 12312 tablet_replica.cc:330] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.973541 12312 raft_consensus.cc:2229] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.973888 12312 raft_consensus.cc:2258] T a484222860f74f7284dbc7fdec58c36f P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.975594 12312 tablet_replica.cc:330] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.976037 12312 raft_consensus.cc:2229] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.976673 12312 raft_consensus.cc:2258] T 3f33b65ff9684d38a7671f1f88e0c190 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.978143 12312 tablet_replica.cc:330] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.978570 12312 raft_consensus.cc:2229] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.978905 12312 raft_consensus.cc:2258] T b05bc22dbb28441cb642e218aa692971 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.980476 12312 tablet_replica.cc:330] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.980911 12312 raft_consensus.cc:2229] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.981253 12312 raft_consensus.cc:2258] T 05c22dbc72b046788317d845ec3ef7b7 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.982753 12312 tablet_replica.cc:330] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.983206 12312 raft_consensus.cc:2229] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.983567 12312 raft_consensus.cc:2258] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.984989 12312 tablet_replica.cc:330] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.985396 12312 raft_consensus.cc:2229] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.985759 12312 raft_consensus.cc:2258] T c088bd75653841da80100a14604827b9 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.987267 12312 tablet_replica.cc:330] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.987668 12312 raft_consensus.cc:2229] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 5 LEADER]: Raft consensus shutting down.
I20240503 08:04:27.988312 12312 raft_consensus.cc:2258] T 7396a6e047f14194afdfaa32186a8858 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:27.989707 12312 tablet_replica.cc:330] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9: stopping tablet replica
I20240503 08:04:27.990095 12312 raft_consensus.cc:2229] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:27.990433 12312 raft_consensus.cc:2258] T 916c02f4a324485d883ead3fdd81f7e1 P 4172ea1e5a3e4cd2b3aa51f3cee814c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.035444 12312 tablet_server.cc:194] TabletServer@127.12.6.2:0 shutdown complete.
W20240503 08:04:28.040361 13081 consensus_peers.cc:485] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c -> Peer 87a73f1536014d609105112116c0e957 (127.12.6.1:34959): Couldn't send request to peer 87a73f1536014d609105112116c0e957. Status: Network error: Client connection negotiation failed: client connection to 127.12.6.1:34959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:28.060945 12312 tablet_server.cc:178] TabletServer@127.12.6.3:0 shutting down...
I20240503 08:04:28.088133 12312 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 08:04:28.089133 12312 tablet_replica.cc:330] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.090037 12312 raft_consensus.cc:2229] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.090711 12312 raft_consensus.cc:2258] T 72e6c2a47f6243ae9d98439b9cb5ff52 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.092785 12312 tablet_replica.cc:330] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.093415 12312 raft_consensus.cc:2229] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.093940 12312 raft_consensus.cc:2258] T 05c22dbc72b046788317d845ec3ef7b7 P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.095896 12312 tablet_replica.cc:330] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.096395 12312 raft_consensus.cc:2229] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.096915 12312 raft_consensus.cc:2258] T 5c013c695821420597e5385fd5a679a1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.098615 12312 tablet_replica.cc:330] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.099280 12312 raft_consensus.cc:2229] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.099789 12312 raft_consensus.cc:2258] T 9aa48fa501614e9d88dababf62e20b36 P 6cfd1f6d1cef47349921f57394a0212c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.101785 12312 tablet_replica.cc:330] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.102506 12312 raft_consensus.cc:2229] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.102982 12312 raft_consensus.cc:2258] T 3f33b65ff9684d38a7671f1f88e0c190 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.104692 12312 tablet_replica.cc:330] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.105131 12312 raft_consensus.cc:2229] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 1 LEADER]: Raft consensus shutting down.
I20240503 08:04:28.106168 12312 raft_consensus.cc:2258] T a484222860f74f7284dbc7fdec58c36f P 6cfd1f6d1cef47349921f57394a0212c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.108705 12312 tablet_replica.cc:330] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.109225 12312 raft_consensus.cc:2229] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.109664 12312 raft_consensus.cc:2258] T c088bd75653841da80100a14604827b9 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.111548 12312 tablet_replica.cc:330] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.111992 12312 raft_consensus.cc:2229] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 5 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:28.112474 12312 raft_consensus.cc:2258] T 7396a6e047f14194afdfaa32186a8858 P 6cfd1f6d1cef47349921f57394a0212c [term 5 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.114506 12312 tablet_replica.cc:330] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.114950 12312 raft_consensus.cc:2229] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:04:28.115978 12312 raft_consensus.cc:2258] T 916c02f4a324485d883ead3fdd81f7e1 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.117672 12312 tablet_replica.cc:330] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c: stopping tablet replica
I20240503 08:04:28.118106 12312 raft_consensus.cc:2229] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 3 LEADER]: Raft consensus shutting down.
I20240503 08:04:28.119104 12312 raft_consensus.cc:2258] T b05bc22dbb28441cb642e218aa692971 P 6cfd1f6d1cef47349921f57394a0212c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.151360 12312 tablet_server.cc:194] TabletServer@127.12.6.3:0 shutdown complete.
I20240503 08:04:28.170719 12312 master.cc:537] Master@127.12.6.62:38357 shutting down...
I20240503 08:04:28.195703 12312 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 1 LEADER]: Raft consensus shutting down.
I20240503 08:04:28.196350 12312 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:28.196650 12312 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70ddc3ee2b3341038f11469260b400bb: stopping tablet replica
I20240503 08:04:28.214452 12312 master.cc:552] Master@127.12.6.62:38357 shutdown complete.
4172ea1e5a3e4cd2b3aa51f3cee814c9 4
6cfd1f6d1cef47349921f57394a0212c 1
87a73f1536014d609105112116c0e957 5
4172ea1e5a3e4cd2b3aa51f3cee814c9 4
6cfd1f6d1cef47349921f57394a0212c 3
87a73f1536014d609105112116c0e957 3
4172ea1e5a3e4cd2b3aa51f3cee814c9 1
6cfd1f6d1cef47349921f57394a0212c 1
87a73f1536014d609105112116c0e957 8
4172ea1e5a3e4cd2b3aa51f3cee814c9 3
6cfd1f6d1cef47349921f57394a0212c 3
87a73f1536014d609105112116c0e957 4
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28855 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20240503 08:04:28.258412 12312 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.6.62:35173
I20240503 08:04:28.259565 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:04:28.266934 13344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.267917 13345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.268915 13347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:04:28.269804 12312 server_base.cc:1034] running on GCE node
I20240503 08:04:28.270777 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:04:28.270980 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:04:28.271157 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723468271113 us; error 0 us; skew 500 ppm
I20240503 08:04:28.271680 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:04:28.274281 12312 webserver.cc:460] Webserver started at http://127.12.6.62:39035/ using document root <none> and password file <none>
I20240503 08:04:28.274776 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:04:28.274976 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:04:28.275543 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:04:28.276830 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/master-0-root/instance:
uuid: "6d0a070b7a094d908809612e09767ad2"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.281590 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20240503 08:04:28.284969 13352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.285725 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240503 08:04:28.286021 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/master-0-root
uuid: "6d0a070b7a094d908809612e09767ad2"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.286309 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:04:28.317863 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:04:28.319175 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:04:28.360163 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.62:35173
I20240503 08:04:28.360325 13403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.62:35173 every 8 connection(s)
I20240503 08:04:28.364408 13404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:28.374346 13404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2: Bootstrap starting.
I20240503 08:04:28.378533 13404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:28.383010 13404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2: No bootstrap required, opened a new log
I20240503 08:04:28.385089 13404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a070b7a094d908809612e09767ad2" member_type: VOTER }
I20240503 08:04:28.385583 13404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:28.385826 13404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0a070b7a094d908809612e09767ad2, State: Initialized, Role: FOLLOWER
I20240503 08:04:28.386456 13404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a070b7a094d908809612e09767ad2" member_type: VOTER }
I20240503 08:04:28.386911 13404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 08:04:28.387219 13404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 08:04:28.387472 13404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:28.391635 13404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a070b7a094d908809612e09767ad2" member_type: VOTER }
I20240503 08:04:28.392169 13404 leader_election.cc:302] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d0a070b7a094d908809612e09767ad2; no voters:
I20240503 08:04:28.393258 13404 leader_election.cc:288] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 08:04:28.393657 13407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:28.394984 13407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [term 1 LEADER]: Becoming Leader. State: Replica: 6d0a070b7a094d908809612e09767ad2, State: Running, Role: LEADER
I20240503 08:04:28.395641 13407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a070b7a094d908809612e09767ad2" member_type: VOTER }
I20240503 08:04:28.396124 13404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 08:04:28.398551 13408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d0a070b7a094d908809612e09767ad2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a070b7a094d908809612e09767ad2" member_type: VOTER } }
I20240503 08:04:28.399417 13408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [sys.catalog]: This master's current role is: LEADER
I20240503 08:04:28.398561 13409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d0a070b7a094d908809612e09767ad2. Latest consensus state: current_term: 1 leader_uuid: "6d0a070b7a094d908809612e09767ad2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a070b7a094d908809612e09767ad2" member_type: VOTER } }
I20240503 08:04:28.400719 13409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2 [sys.catalog]: This master's current role is: LEADER
I20240503 08:04:28.403666 13413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 08:04:28.411233 13413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 08:04:28.412233 12312 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240503 08:04:28.419896 13413 catalog_manager.cc:1347] Generated new cluster ID: c17e3b0d196e410ea9d39925e3a1ee4a
I20240503 08:04:28.420185 13413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 08:04:28.435067 13413 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 08:04:28.436371 13413 catalog_manager.cc:1504] Loading token signing keys...
I20240503 08:04:28.448495 13413 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 6d0a070b7a094d908809612e09767ad2: Generated new TSK 0
I20240503 08:04:28.449095 13413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 08:04:28.479782 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:04:28.486833 13425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.488054 13426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.488962 13428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:04:28.489699 12312 server_base.cc:1034] running on GCE node
I20240503 08:04:28.490545 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:04:28.490729 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:04:28.490892 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723468490871 us; error 0 us; skew 500 ppm
I20240503 08:04:28.491462 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:04:28.493794 12312 webserver.cc:460] Webserver started at http://127.12.6.1:42375/ using document root <none> and password file <none>
I20240503 08:04:28.494243 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:04:28.494419 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:04:28.494669 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:04:28.495818 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root/instance:
uuid: "f2462cc996fb441a847521d550b300a4"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.500305 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20240503 08:04:28.503567 13433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.504303 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20240503 08:04:28.504617 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root
uuid: "f2462cc996fb441a847521d550b300a4"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.504901 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:04:28.527861 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:04:28.529033 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:04:28.530431 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:04:28.532681 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:04:28.532881 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.533116 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:04:28.533289 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.577371 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.1:34427
I20240503 08:04:28.577399 13495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.1:34427 every 8 connection(s)
I20240503 08:04:28.583457 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:04:28.592031 13500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.593089 13501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.594502 13503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:04:28.596283 13496 heartbeater.cc:347] Connected to a master server at 127.12.6.62:35173
I20240503 08:04:28.596283 12312 server_base.cc:1034] running on GCE node
I20240503 08:04:28.596907 13496 heartbeater.cc:464] Registering TS with master...
I20240503 08:04:28.598006 13496 heartbeater.cc:511] Master 127.12.6.62:35173 requested a full tablet report, sending...
I20240503 08:04:28.598426 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:04:28.598711 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:04:28.598923 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723468598905 us; error 0 us; skew 500 ppm
I20240503 08:04:28.599762 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:04:28.600554 13369 ts_manager.cc:194] Registered new tserver with Master: f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:28.602519 12312 webserver.cc:460] Webserver started at http://127.12.6.2:37757/ using document root <none> and password file <none>
I20240503 08:04:28.602658 13369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56434
I20240503 08:04:28.603061 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:04:28.603346 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:04:28.603616 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:04:28.605250 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root/instance:
uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.609858 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20240503 08:04:28.613111 13508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.613886 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240503 08:04:28.614169 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root
uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.614444 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:04:28.634748 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:04:28.636018 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:04:28.637421 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:04:28.639729 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:04:28.639940 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.640172 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:04:28.640324 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.679587 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.2:44917
I20240503 08:04:28.679693 13570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.2:44917 every 8 connection(s)
I20240503 08:04:28.684262 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:04:28.692477 13574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:28.693897 13575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:04:28.696492 13571 heartbeater.cc:347] Connected to a master server at 127.12.6.62:35173
I20240503 08:04:28.696986 13571 heartbeater.cc:464] Registering TS with master...
I20240503 08:04:28.697889 13571 heartbeater.cc:511] Master 127.12.6.62:35173 requested a full tablet report, sending...
I20240503 08:04:28.698647 12312 server_base.cc:1034] running on GCE node
W20240503 08:04:28.699579 13578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:04:28.699981 13369 ts_manager.cc:194] Registered new tserver with Master: ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:28.700416 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:04:28.700619 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:04:28.700781 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723468700763 us; error 0 us; skew 500 ppm
I20240503 08:04:28.701344 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:04:28.701521 13369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56446
I20240503 08:04:28.704201 12312 webserver.cc:460] Webserver started at http://127.12.6.3:39363/ using document root <none> and password file <none>
I20240503 08:04:28.704732 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:04:28.704926 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:04:28.705222 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:04:28.706485 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root/instance:
uuid: "cf70d9fd04894ea3812767cdb12156c5"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.710812 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20240503 08:04:28.713982 13582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.714720 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240503 08:04:28.715021 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root
uuid: "cf70d9fd04894ea3812767cdb12156c5"
format_stamp: "Formatted at 2024-05-03 08:04:28 on dist-test-slave-6qcz"
I20240503 08:04:28.715368 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:04:28.741112 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:04:28.742261 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:04:28.743695 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:04:28.745905 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:04:28.746114 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.746340 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:04:28.746496 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:28.785034 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.3:33827
I20240503 08:04:28.785097 13644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.3:33827 every 8 connection(s)
I20240503 08:04:28.798318 13645 heartbeater.cc:347] Connected to a master server at 127.12.6.62:35173
I20240503 08:04:28.798674 13645 heartbeater.cc:464] Registering TS with master...
I20240503 08:04:28.799381 13645 heartbeater.cc:511] Master 127.12.6.62:35173 requested a full tablet report, sending...
I20240503 08:04:28.801445 13369 ts_manager.cc:194] Registered new tserver with Master: cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:28.801652 12312 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013755465s
I20240503 08:04:28.803364 13369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56456
I20240503 08:04:28.804759 12312 test_util.cc:268] Using random seed: 725549696
I20240503 08:04:28.828613 13369 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56470:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 08:04:28.834599 13369 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 08:04:29.030273 13608 tablet_service.cc:1467] Processing CreateTablet for tablet 6f02eda12b9544888367d6206e3b41dd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240503 08:04:29.030813 13607 tablet_service.cc:1467] Processing CreateTablet for tablet a2325f0703f147ffabe6be77e2db6433 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240503 08:04:29.031881 13608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f02eda12b9544888367d6206e3b41dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.032708 13461 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6dbe71928443b09931a1b356044911 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240503 08:04:29.033082 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 768f7ed6c5cc40bd947c30cf5450c8f1 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240503 08:04:29.034489 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6dbe71928443b09931a1b356044911. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.034482 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768f7ed6c5cc40bd947c30cf5450c8f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.036336 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 911fa6daf5354732a1c3741fe6da0fa7 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240503 08:04:29.036765 13460 tablet_service.cc:1467] Processing CreateTablet for tablet 911fa6daf5354732a1c3741fe6da0fa7 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240503 08:04:29.038362 13460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 911fa6daf5354732a1c3741fe6da0fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.038614 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2325f0703f147ffabe6be77e2db6433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.038231 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 768f7ed6c5cc40bd947c30cf5450c8f1 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240503 08:04:29.039604 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768f7ed6c5cc40bd947c30cf5450c8f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.039731 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 911fa6daf5354732a1c3741fe6da0fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.044426 13456 tablet_service.cc:1467] Processing CreateTablet for tablet b181d58f5aef42679e79edcd3b81c20b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240503 08:04:29.045868 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b181d58f5aef42679e79edcd3b81c20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.048341 13458 tablet_service.cc:1467] Processing CreateTablet for tablet a2325f0703f147ffabe6be77e2db6433 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240503 08:04:29.049732 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2325f0703f147ffabe6be77e2db6433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.052471 13459 tablet_service.cc:1467] Processing CreateTablet for tablet 6f02eda12b9544888367d6206e3b41dd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240503 08:04:29.053905 13459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f02eda12b9544888367d6206e3b41dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.038232 13610 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6dbe71928443b09931a1b356044911 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240503 08:04:29.056466 13610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6dbe71928443b09931a1b356044911. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.089041 13605 tablet_service.cc:1467] Processing CreateTablet for tablet b181d58f5aef42679e79edcd3b81c20b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240503 08:04:29.090530 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b181d58f5aef42679e79edcd3b81c20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.092204 13660 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:29.101995 13455 tablet_service.cc:1467] Processing CreateTablet for tablet 07fd7638c4454552a7aeb0aa85bef63b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240503 08:04:29.103839 13455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07fd7638c4454552a7aeb0aa85bef63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.115154 13660 tablet_bootstrap.cc:654] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.115278 13661 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:29.115761 13536 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6dbe71928443b09931a1b356044911 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240503 08:04:29.116294 13535 tablet_service.cc:1467] Processing CreateTablet for tablet 911fa6daf5354732a1c3741fe6da0fa7 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240503 08:04:29.117094 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6dbe71928443b09931a1b356044911. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.117918 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 911fa6daf5354732a1c3741fe6da0fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.121327 13661 tablet_bootstrap.cc:654] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.126780 13604 tablet_service.cc:1467] Processing CreateTablet for tablet 07fd7638c4454552a7aeb0aa85bef63b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240503 08:04:29.127804 13461 tablet_service.cc:1467] Processing CreateTablet for tablet a6234f05657b4c0fbb21339b1a63d571 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240503 08:04:29.128289 13604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07fd7638c4454552a7aeb0aa85bef63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.128294 13533 tablet_service.cc:1467] Processing CreateTablet for tablet a2325f0703f147ffabe6be77e2db6433 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240503 08:04:29.129046 13534 tablet_service.cc:1467] Processing CreateTablet for tablet 6f02eda12b9544888367d6206e3b41dd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240503 08:04:29.129846 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2325f0703f147ffabe6be77e2db6433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.130511 13534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f02eda12b9544888367d6206e3b41dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.140916 13603 tablet_service.cc:1467] Processing CreateTablet for tablet a6234f05657b4c0fbb21339b1a63d571 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240503 08:04:29.141605 13454 tablet_service.cc:1467] Processing CreateTablet for tablet ff2dd11388594890a81f290c0bac6840 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240503 08:04:29.142405 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6234f05657b4c0fbb21339b1a63d571. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.142876 13454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2dd11388594890a81f290c0bac6840. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.151917 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6234f05657b4c0fbb21339b1a63d571. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.155282 13532 tablet_service.cc:1467] Processing CreateTablet for tablet 768f7ed6c5cc40bd947c30cf5450c8f1 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240503 08:04:29.163861 13456 tablet_service.cc:1467] Processing CreateTablet for tablet d04ac96e14f7469dba60c7f68d25e882 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240503 08:04:29.164438 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768f7ed6c5cc40bd947c30cf5450c8f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.158224 13531 tablet_service.cc:1467] Processing CreateTablet for tablet b181d58f5aef42679e79edcd3b81c20b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240503 08:04:29.168118 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b181d58f5aef42679e79edcd3b81c20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.158977 13530 tablet_service.cc:1467] Processing CreateTablet for tablet 07fd7638c4454552a7aeb0aa85bef63b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240503 08:04:29.161388 13529 tablet_service.cc:1467] Processing CreateTablet for tablet a6234f05657b4c0fbb21339b1a63d571 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240503 08:04:29.173089 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6234f05657b4c0fbb21339b1a63d571. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.175791 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07fd7638c4454552a7aeb0aa85bef63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.177228 13460 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1a896bcf18443fb65ff2a59c8dfe3f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240503 08:04:29.180019 13458 tablet_service.cc:1467] Processing CreateTablet for tablet e3ca0d7dd59d4881b4c9e6a2a6a8a505 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240503 08:04:29.175319 13455 tablet_service.cc:1467] Processing CreateTablet for tablet 161465c92bbf4897bc9944c2b9869a9a (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240503 08:04:29.190008 13660 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:29.189952 13610 tablet_service.cc:1467] Processing CreateTablet for tablet d04ac96e14f7469dba60c7f68d25e882 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240503 08:04:29.191013 13660 ts_tablet_manager.cc:1367] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.099s user 0.015s sys 0.004s
I20240503 08:04:29.191201 13459 tablet_service.cc:1467] Processing CreateTablet for tablet ae6f8cbdf9a94be9a490fc84cf1c3d7e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240503 08:04:29.191762 13610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04ac96e14f7469dba60c7f68d25e882. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.193845 13660 raft_consensus.cc:348] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.194599 13660 raft_consensus.cc:374] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:29.194878 13660 raft_consensus.cc:729] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:29.197902 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ca0d7dd59d4881b4c9e6a2a6a8a505. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.198557 13460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1a896bcf18443fb65ff2a59c8dfe3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.199673 13528 tablet_service.cc:1467] Processing CreateTablet for tablet ff2dd11388594890a81f290c0bac6840 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240503 08:04:29.200981 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2dd11388594890a81f290c0bac6840. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.207279 13527 tablet_service.cc:1467] Processing CreateTablet for tablet d04ac96e14f7469dba60c7f68d25e882 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240503 08:04:29.208767 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04ac96e14f7469dba60c7f68d25e882. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.209875 13453 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1b40442f084e05b58edb1d96de3fa3 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240503 08:04:29.212416 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 161465c92bbf4897bc9944c2b9869a9a (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240503 08:04:29.213711 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161465c92bbf4897bc9944c2b9869a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.213390 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1a896bcf18443fb65ff2a59c8dfe3f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240503 08:04:29.214766 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1a896bcf18443fb65ff2a59c8dfe3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.215611 13602 tablet_service.cc:1467] Processing CreateTablet for tablet 75f41e425e564506958c79d20cc661e1 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240503 08:04:29.215497 13608 tablet_service.cc:1467] Processing CreateTablet for tablet e3ca0d7dd59d4881b4c9e6a2a6a8a505 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240503 08:04:29.213348 13609 tablet_service.cc:1467] Processing CreateTablet for tablet ae6f8cbdf9a94be9a490fc84cf1c3d7e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240503 08:04:29.207195 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 75f41e425e564506958c79d20cc661e1 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240503 08:04:29.216264 13452 tablet_service.cc:1467] Processing CreateTablet for tablet ef1e609185a44d14a5473954db083b1a (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240503 08:04:29.221451 13602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f41e425e564506958c79d20cc661e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.223757 13607 tablet_service.cc:1467] Processing CreateTablet for tablet ff2dd11388594890a81f290c0bac6840 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240503 08:04:29.225016 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2dd11388594890a81f290c0bac6840. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.229202 13601 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1b40442f084e05b58edb1d96de3fa3 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240503 08:04:29.230780 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1b40442f084e05b58edb1d96de3fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.232249 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6f8cbdf9a94be9a490fc84cf1c3d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.235255 13660 consensus_queue.cc:260] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.242417 13534 tablet_service.cc:1467] Processing CreateTablet for tablet 161465c92bbf4897bc9944c2b9869a9a (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240503 08:04:29.244323 13608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ca0d7dd59d4881b4c9e6a2a6a8a505. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.253846 13535 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1a896bcf18443fb65ff2a59c8dfe3f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240503 08:04:29.255295 13459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6f8cbdf9a94be9a490fc84cf1c3d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.255999 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f41e425e564506958c79d20cc661e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.256867 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04ac96e14f7469dba60c7f68d25e882. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.257509 13453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1b40442f084e05b58edb1d96de3fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.259461 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e609185a44d14a5473954db083b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.260823 13603 tablet_service.cc:1467] Processing CreateTablet for tablet ef1e609185a44d14a5473954db083b1a (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240503 08:04:29.262122 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e609185a44d14a5473954db083b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.260824 13454 tablet_service.cc:1467] Processing CreateTablet for tablet 248eecaf8416442794c9558a2d73bdc9 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240503 08:04:29.265589 13661 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:29.266135 13661 ts_tablet_manager.cc:1367] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.151s user 0.008s sys 0.014s
I20240503 08:04:29.271723 13455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161465c92bbf4897bc9944c2b9869a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.277396 13645 heartbeater.cc:503] Master 127.12.6.62:35173 was elected leader, sending a full tablet report...
I20240503 08:04:29.277812 13660 ts_tablet_manager.cc:1398] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.086s user 0.004s sys 0.004s
I20240503 08:04:29.278728 13660 tablet_bootstrap.cc:492] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:29.285930 13665 tablet_bootstrap.cc:492] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:29.286346 13454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248eecaf8416442794c9558a2d73bdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.286463 13661 raft_consensus.cc:348] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.287464 13661 raft_consensus.cc:374] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:29.287910 13661 raft_consensus.cc:729] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:29.294026 13458 tablet_service.cc:1467] Processing CreateTablet for tablet f63fcf8624f04059b2dd161f9ae967ea (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240503 08:04:29.301626 13536 tablet_service.cc:1467] Processing CreateTablet for tablet ae6f8cbdf9a94be9a490fc84cf1c3d7e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240503 08:04:29.315642 13603 tablet_service.cc:1467] Processing CreateTablet for tablet 248eecaf8416442794c9558a2d73bdc9 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240503 08:04:29.313865 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63fcf8624f04059b2dd161f9ae967ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.317812 13610 tablet_service.cc:1467] Processing CreateTablet for tablet 09fca1878b3e47838c773a0ea155e22b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240503 08:04:29.315721 13609 tablet_service.cc:1467] Processing CreateTablet for tablet f63fcf8624f04059b2dd161f9ae967ea (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240503 08:04:29.318219 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248eecaf8416442794c9558a2d73bdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.284797 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1a896bcf18443fb65ff2a59c8dfe3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.325667 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6f8cbdf9a94be9a490fc84cf1c3d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.325795 13530 tablet_service.cc:1467] Processing CreateTablet for tablet 4b1b40442f084e05b58edb1d96de3fa3 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240503 08:04:29.327741 13661 consensus_queue.cc:260] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.328300 13460 tablet_service.cc:1467] Processing CreateTablet for tablet 09fca1878b3e47838c773a0ea155e22b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240503 08:04:29.329913 13460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09fca1878b3e47838c773a0ea155e22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.336898 13533 tablet_service.cc:1467] Processing CreateTablet for tablet 75f41e425e564506958c79d20cc661e1 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240503 08:04:29.339448 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b1b40442f084e05b58edb1d96de3fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.346688 13531 tablet_service.cc:1467] Processing CreateTablet for tablet e3ca0d7dd59d4881b4c9e6a2a6a8a505 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240503 08:04:29.348213 13532 tablet_service.cc:1467] Processing CreateTablet for tablet ef1e609185a44d14a5473954db083b1a (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240503 08:04:29.353991 13664 raft_consensus.cc:482] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:29.354602 13664 raft_consensus.cc:504] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.361261 13610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09fca1878b3e47838c773a0ea155e22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.361979 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63fcf8624f04059b2dd161f9ae967ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.348765 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ca0d7dd59d4881b4c9e6a2a6a8a505. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 08:04:29.393862 13355 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.6.1:34427, user_credentials={real_user=slave}} blocked reactor thread for 73862.3us
W20240503 08:04:29.404726 13356 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 77973.2us
I20240503 08:04:29.411532 13664 leader_election.cc:288] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:29.416548 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f41e425e564506958c79d20cc661e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.422870 13534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161465c92bbf4897bc9944c2b9869a9a. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 08:04:29.439394 13353 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 103271us
W20240503 08:04:29.439527 13355 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.6.1:34427, user_credentials={real_user=slave}} blocked reactor thread for 44324.5us
W20240503 08:04:29.440874 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:29.444744 13660 tablet_bootstrap.cc:654] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.455268 13602 tablet_service.cc:1467] Processing CreateTablet for tablet 0786fc1af8ae401ebb5190aa117ad461 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240503 08:04:29.456630 13602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0786fc1af8ae401ebb5190aa117ad461. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.462950 13461 tablet_service.cc:1467] Processing CreateTablet for tablet e8e4a81f1b4b4ec69620d613e4273c73 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240503 08:04:29.464690 13665 tablet_bootstrap.cc:654] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.465207 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e4a81f1b4b4ec69620d613e4273c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.468166 13459 tablet_service.cc:1467] Processing CreateTablet for tablet 0786fc1af8ae401ebb5190aa117ad461 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240503 08:04:29.469453 13459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0786fc1af8ae401ebb5190aa117ad461. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.480345 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e609185a44d14a5473954db083b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.483441 13529 tablet_service.cc:1467] Processing CreateTablet for tablet 248eecaf8416442794c9558a2d73bdc9 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240503 08:04:29.484861 13601 tablet_service.cc:1467] Processing CreateTablet for tablet e8e4a81f1b4b4ec69620d613e4273c73 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240503 08:04:29.485335 13660 tablet_bootstrap.cc:492] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:29.500676 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e4a81f1b4b4ec69620d613e4273c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.504351 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 153d55380cbd4262965848a39a01e17d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240503 08:04:29.505803 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153d55380cbd4262965848a39a01e17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.509822 13528 tablet_service.cc:1467] Processing CreateTablet for tablet f63fcf8624f04059b2dd161f9ae967ea (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240503 08:04:29.512300 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 153d55380cbd4262965848a39a01e17d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240503 08:04:29.514232 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153d55380cbd4262965848a39a01e17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.515472 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248eecaf8416442794c9558a2d73bdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.523216 13456 tablet_service.cc:1467] Processing CreateTablet for tablet def4168780ca4a61aab9436355acca14 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240503 08:04:29.524991 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def4168780ca4a61aab9436355acca14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.498708 13660 ts_tablet_manager.cc:1367] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.220s user 0.012s sys 0.008s
I20240503 08:04:29.518283 13452 tablet_service.cc:1467] Processing CreateTablet for tablet 0f58e3d02a7d451c90bc97b49b1f83dd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240503 08:04:29.528476 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f58e3d02a7d451c90bc97b49b1f83dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.528561 13661 ts_tablet_manager.cc:1398] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.262s user 0.035s sys 0.034s
I20240503 08:04:29.529362 13496 heartbeater.cc:503] Master 127.12.6.62:35173 was elected leader, sending a full tablet report...
I20240503 08:04:29.530040 13660 raft_consensus.cc:348] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.530874 13660 raft_consensus.cc:374] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:29.532407 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63fcf8624f04059b2dd161f9ae967ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.534902 13661 tablet_bootstrap.cc:492] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:29.479995 13608 tablet_service.cc:1467] Processing CreateTablet for tablet def4168780ca4a61aab9436355acca14 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240503 08:04:29.540829 13666 raft_consensus.cc:482] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:29.541348 13666 raft_consensus.cc:504] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.536932 13660 raft_consensus.cc:729] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:29.571718 13607 tablet_service.cc:1467] Processing CreateTablet for tablet 0f58e3d02a7d451c90bc97b49b1f83dd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240503 08:04:29.575645 13666 leader_election.cc:288] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:29.576400 13608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def4168780ca4a61aab9436355acca14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.578521 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f58e3d02a7d451c90bc97b49b1f83dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.599485 13665 tablet_bootstrap.cc:492] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:29.599999 13665 ts_tablet_manager.cc:1367] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.314s user 0.017s sys 0.035s
I20240503 08:04:29.604089 13665 raft_consensus.cc:348] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.606101 13665 raft_consensus.cc:374] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:29.606842 13665 raft_consensus.cc:729] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:29.608100 13665 consensus_queue.cc:260] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:29.621855 13356 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 50784.3us
I20240503 08:04:29.622522 13527 tablet_service.cc:1467] Processing CreateTablet for tablet 0786fc1af8ae401ebb5190aa117ad461 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240503 08:04:29.623838 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0786fc1af8ae401ebb5190aa117ad461. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.627976 13535 tablet_service.cc:1467] Processing CreateTablet for tablet 153d55380cbd4262965848a39a01e17d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240503 08:04:29.629268 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153d55380cbd4262965848a39a01e17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.634747 13661 tablet_bootstrap.cc:654] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.651773 13530 tablet_service.cc:1467] Processing CreateTablet for tablet def4168780ca4a61aab9436355acca14 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240503 08:04:29.653184 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def4168780ca4a61aab9436355acca14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.659999 13533 tablet_service.cc:1467] Processing CreateTablet for tablet e8e4a81f1b4b4ec69620d613e4273c73 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240503 08:04:29.661356 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e4a81f1b4b4ec69620d613e4273c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.662480 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:29.668473 13583 leader_election.cc:341] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:29.674688 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:29.675081 13532 tablet_service.cc:1467] Processing CreateTablet for tablet 0f58e3d02a7d451c90bc97b49b1f83dd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240503 08:04:29.679664 13459 tablet_service.cc:1467] Processing CreateTablet for tablet 0e729d022cc745f6b21dc4320432c817 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240503 08:04:29.682392 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f58e3d02a7d451c90bc97b49b1f83dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.683001 13459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e729d022cc745f6b21dc4320432c817. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 08:04:29.684149 13434 leader_election.cc:341] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:29.686784 13453 tablet_service.cc:1467] Processing CreateTablet for tablet ad7e36891053477ba558573130590a0f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240503 08:04:29.688545 13453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7e36891053477ba558573130590a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.692427 13531 tablet_service.cc:1467] Processing CreateTablet for tablet 09fca1878b3e47838c773a0ea155e22b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240503 08:04:29.694126 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09fca1878b3e47838c773a0ea155e22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.708421 13571 heartbeater.cc:503] Master 127.12.6.62:35173 was elected leader, sending a full tablet report...
I20240503 08:04:29.709075 13534 tablet_service.cc:1467] Processing CreateTablet for tablet ad7e36891053477ba558573130590a0f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240503 08:04:29.710855 13534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7e36891053477ba558573130590a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.718350 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:29.759655 13471 raft_consensus.cc:2454] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:29.595093 13660 consensus_queue.cc:260] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.750921 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 51cb053f76ea42bd8ed2a6c66895593d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240503 08:04:29.751956 13461 tablet_service.cc:1467] Processing CreateTablet for tablet f190cd50f6c2434ebf6d8e0d490eafe7 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240503 08:04:29.752785 13458 tablet_service.cc:1467] Processing CreateTablet for tablet 805134c1c2c84e2eab2b59a184b7abd4 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240503 08:04:29.761615 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51cb053f76ea42bd8ed2a6c66895593d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.762348 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 805134c1c2c84e2eab2b59a184b7abd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.763053 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f190cd50f6c2434ebf6d8e0d490eafe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.763234 13601 tablet_service.cc:1467] Processing CreateTablet for tablet 0e729d022cc745f6b21dc4320432c817 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240503 08:04:29.764464 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e729d022cc745f6b21dc4320432c817. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.767483 13664 raft_consensus.cc:482] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:29.767997 13664 raft_consensus.cc:504] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.773545 13664 leader_election.cc:288] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:29.773911 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "911fa6daf5354732a1c3741fe6da0fa7" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:29.778985 13603 tablet_service.cc:1467] Processing CreateTablet for tablet 51cb053f76ea42bd8ed2a6c66895593d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240503 08:04:29.779799 13602 tablet_service.cc:1467] Processing CreateTablet for tablet f190cd50f6c2434ebf6d8e0d490eafe7 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240503 08:04:29.779963 13529 tablet_service.cc:1467] Processing CreateTablet for tablet 0e729d022cc745f6b21dc4320432c817 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240503 08:04:29.780354 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51cb053f76ea42bd8ed2a6c66895593d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.781037 13602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f190cd50f6c2434ebf6d8e0d490eafe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.783167 13528 tablet_service.cc:1467] Processing CreateTablet for tablet 51cb053f76ea42bd8ed2a6c66895593d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240503 08:04:29.787710 13585 leader_election.cc:302] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:29.788631 13664 raft_consensus.cc:2781] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:29.788996 13664 raft_consensus.cc:482] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:29.789341 13664 raft_consensus.cc:3037] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:29.789454 13585 leader_election.cc:341] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:29.794013 13660 ts_tablet_manager.cc:1398] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.266s user 0.009s sys 0.000s
I20240503 08:04:29.794662 13665 ts_tablet_manager.cc:1398] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.194s user 0.038s sys 0.080s
I20240503 08:04:29.801086 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 805134c1c2c84e2eab2b59a184b7abd4 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240503 08:04:29.802466 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 805134c1c2c84e2eab2b59a184b7abd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.802225 13536 tablet_service.cc:1467] Processing CreateTablet for tablet f190cd50f6c2434ebf6d8e0d490eafe7 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240503 08:04:29.795006 13660 tablet_bootstrap.cc:492] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:29.807889 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "911fa6daf5354732a1c3741fe6da0fa7" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:29.809401 13583 leader_election.cc:341] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:29.809863 13583 leader_election.cc:302] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:29.761554 13610 tablet_service.cc:1467] Processing CreateTablet for tablet ad7e36891053477ba558573130590a0f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240503 08:04:29.811571 13527 tablet_service.cc:1467] Processing CreateTablet for tablet 805134c1c2c84e2eab2b59a184b7abd4 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240503 08:04:29.812860 13677 raft_consensus.cc:2726] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:29.811210 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51cb053f76ea42bd8ed2a6c66895593d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.814244 13610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7e36891053477ba558573130590a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.815913 13665 tablet_bootstrap.cc:492] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:29.826092 13664 raft_consensus.cc:504] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.831154 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 805134c1c2c84e2eab2b59a184b7abd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.833315 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f190cd50f6c2434ebf6d8e0d490eafe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.834131 13660 tablet_bootstrap.cc:654] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.844017 13665 tablet_bootstrap.cc:654] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:29.847383 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e729d022cc745f6b21dc4320432c817. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.851610 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:29.852494 13620 raft_consensus.cc:2379] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
I20240503 08:04:29.854214 13603 tablet_service.cc:1467] Processing CreateTablet for tablet cac21dd494074d0fa7690d8d182585f4 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240503 08:04:29.855593 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac21dd494074d0fa7690d8d182585f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.856371 13437 leader_election.cc:302] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:29.857407 13671 raft_consensus.cc:3037] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:29.858539 13461 tablet_service.cc:1467] Processing CreateTablet for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240503 08:04:29.859850 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.861110 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbbe543644940cdab442cd183707b91 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240503 08:04:29.862380 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbbe543644940cdab442cd183707b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.864135 13452 tablet_service.cc:1467] Processing CreateTablet for tablet e833906e87754702bba4e5dbde972f86 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240503 08:04:29.865412 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e833906e87754702bba4e5dbde972f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.866003 13530 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbbe543644940cdab442cd183707b91 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240503 08:04:29.867273 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbbe543644940cdab442cd183707b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.868556 13671 raft_consensus.cc:2726] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:29.876958 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:29.880517 13459 tablet_service.cc:1467] Processing CreateTablet for tablet cac21dd494074d0fa7690d8d182585f4 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240503 08:04:29.881810 13459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac21dd494074d0fa7690d8d182585f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.879017 13533 tablet_service.cc:1467] Processing CreateTablet for tablet cac21dd494074d0fa7690d8d182585f4 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240503 08:04:29.883500 13532 tablet_service.cc:1467] Processing CreateTablet for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240503 08:04:29.884382 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac21dd494074d0fa7690d8d182585f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.887871 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.888164 13601 tablet_service.cc:1467] Processing CreateTablet for tablet e833906e87754702bba4e5dbde972f86 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240503 08:04:29.890317 13470 raft_consensus.cc:2454] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:29.892191 13458 tablet_service.cc:1467] Processing CreateTablet for tablet 4fde14f2672c4c0bb9d76f225efd306e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240503 08:04:29.892956 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 4fde14f2672c4c0bb9d76f225efd306e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240503 08:04:29.901122 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fde14f2672c4c0bb9d76f225efd306e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.901217 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fde14f2672c4c0bb9d76f225efd306e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.903172 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e833906e87754702bba4e5dbde972f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.896082 13456 tablet_service.cc:1467] Processing CreateTablet for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240503 08:04:29.903986 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbbe543644940cdab442cd183707b91 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240503 08:04:29.905359 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbbe543644940cdab442cd183707b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.905309 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.919517 13660 tablet_bootstrap.cc:492] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:29.920064 13660 ts_tablet_manager.cc:1367] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.125s user 0.015s sys 0.008s
I20240503 08:04:29.924525 13660 raft_consensus.cc:348] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.931272 13585 leader_election.cc:302] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:29.930792 13660 raft_consensus.cc:374] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:29.931888 13660 raft_consensus.cc:729] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:29.932210 13677 raft_consensus.cc:2781] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:29.896068 13608 tablet_service.cc:1467] Processing CreateTablet for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240503 08:04:29.940403 13608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.941428 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
W20240503 08:04:29.943078 13583 leader_election.cc:341] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:29.943619 13607 tablet_service.cc:1467] Processing CreateTablet for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240503 08:04:29.944139 13661 tablet_bootstrap.cc:492] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:29.944145 13664 leader_election.cc:288] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:29.944622 13661 ts_tablet_manager.cc:1367] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.410s user 0.013s sys 0.008s
I20240503 08:04:29.944957 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.956404 13453 tablet_service.cc:1467] Processing CreateTablet for tablet e15d8c1499084cfabc5ed6031b6a22ad (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240503 08:04:29.956836 13534 tablet_service.cc:1467] Processing CreateTablet for tablet e833906e87754702bba4e5dbde972f86 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240503 08:04:29.957751 13453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e15d8c1499084cfabc5ed6031b6a22ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.958112 13534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e833906e87754702bba4e5dbde972f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.959038 13533 tablet_service.cc:1467] Processing CreateTablet for tablet 4fde14f2672c4c0bb9d76f225efd306e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240503 08:04:29.960299 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fde14f2672c4c0bb9d76f225efd306e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.972991 13529 tablet_service.cc:1467] Processing CreateTablet for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240503 08:04:29.974375 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.978722 13601 tablet_service.cc:1467] Processing CreateTablet for tablet e15d8c1499084cfabc5ed6031b6a22ad (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240503 08:04:29.979348 13675 raft_consensus.cc:482] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:29.980067 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e15d8c1499084cfabc5ed6031b6a22ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.979818 13675 raft_consensus.cc:504] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.982300 13675 leader_election.cc:288] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:29.982983 13528 tablet_service.cc:1467] Processing CreateTablet for tablet e15d8c1499084cfabc5ed6031b6a22ad (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240503 08:04:29.990463 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e15d8c1499084cfabc5ed6031b6a22ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.990682 13454 tablet_service.cc:1467] Processing CreateTablet for tablet 8497edff1aaf45e48da16ad73a649c6d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240503 08:04:29.991946 13454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8497edff1aaf45e48da16ad73a649c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.991665 13606 tablet_service.cc:1467] Processing CreateTablet for tablet e04443187a214cc6ad2bd52022d10822 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240503 08:04:29.992931 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e04443187a214cc6ad2bd52022d10822. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.996188 13604 tablet_service.cc:1467] Processing CreateTablet for tablet 8497edff1aaf45e48da16ad73a649c6d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240503 08:04:29.997673 13604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8497edff1aaf45e48da16ad73a649c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.998193 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 4061d55705594102a8b057896c5f86d0 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240503 08:04:29.999430 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4061d55705594102a8b057896c5f86d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.000110 13460 tablet_service.cc:1467] Processing CreateTablet for tablet aae93c20681b4e4fa9b521edbb9e65e0 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240503 08:04:30.001295 13460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aae93c20681b4e4fa9b521edbb9e65e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.020004 13665 tablet_bootstrap.cc:492] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:30.020578 13665 ts_tablet_manager.cc:1367] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.205s user 0.010s sys 0.020s
I20240503 08:04:30.023949 13602 tablet_service.cc:1467] Processing CreateTablet for tablet 4061d55705594102a8b057896c5f86d0 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240503 08:04:30.025381 13602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4061d55705594102a8b057896c5f86d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:29.932735 13660 consensus_queue.cc:260] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:29.990715 13455 tablet_service.cc:1467] Processing CreateTablet for tablet e04443187a214cc6ad2bd52022d10822 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240503 08:04:30.028286 13455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e04443187a214cc6ad2bd52022d10822. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.029351 13660 ts_tablet_manager.cc:1398] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.109s user 0.005s sys 0.002s
I20240503 08:04:30.030750 13660 tablet_bootstrap.cc:492] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:30.032334 13535 tablet_service.cc:1467] Processing CreateTablet for tablet 4061d55705594102a8b057896c5f86d0 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240503 08:04:30.034143 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4061d55705594102a8b057896c5f86d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.043416 13665 raft_consensus.cc:348] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.044251 13665 raft_consensus.cc:374] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.044206 13452 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13da943b8b4caa8da56b29cd48f93f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240503 08:04:30.044643 13665 raft_consensus.cc:729] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.045511 13665 consensus_queue.cc:260] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.053705 13531 tablet_service.cc:1467] Processing CreateTablet for tablet aae93c20681b4e4fa9b521edbb9e65e0 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240503 08:04:30.055538 13665 ts_tablet_manager.cc:1398] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.035s user 0.008s sys 0.000s
I20240503 08:04:30.056512 13661 raft_consensus.cc:348] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.057324 13661 raft_consensus.cc:374] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.057722 13661 raft_consensus.cc:729] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.056533 13665 tablet_bootstrap.cc:492] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:30.059088 13661 consensus_queue.cc:260] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.060045 13530 tablet_service.cc:1467] Processing CreateTablet for tablet e04443187a214cc6ad2bd52022d10822 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240503 08:04:30.061480 13661 ts_tablet_manager.cc:1398] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.116s user 0.007s sys 0.005s
I20240503 08:04:30.062357 13661 tablet_bootstrap.cc:492] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.055030 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aae93c20681b4e4fa9b521edbb9e65e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.008595 13677 raft_consensus.cc:686] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:30.093989 13671 raft_consensus.cc:482] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.095088 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13da943b8b4caa8da56b29cd48f93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.094739 13671 raft_consensus.cc:504] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.097978 13671 leader_election.cc:288] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:30.098598 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e04443187a214cc6ad2bd52022d10822. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.099650 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6dbe71928443b09931a1b356044911" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:30.101595 13437 leader_election.cc:341] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.108793 13665 tablet_bootstrap.cc:654] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.108719 13602 tablet_service.cc:1467] Processing CreateTablet for tablet aae93c20681b4e4fa9b521edbb9e65e0 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240503 08:04:30.123766 13534 tablet_service.cc:1467] Processing CreateTablet for tablet aa42fc3b9d894d398b7002af94aa3f40 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240503 08:04:30.125111 13534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa42fc3b9d894d398b7002af94aa3f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.127106 13455 tablet_service.cc:1467] Processing CreateTablet for tablet aa42fc3b9d894d398b7002af94aa3f40 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240503 08:04:30.127960 13660 tablet_bootstrap.cc:654] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.129310 13536 tablet_service.cc:1467] Processing CreateTablet for tablet 8497edff1aaf45e48da16ad73a649c6d (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240503 08:04:30.129861 13455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa42fc3b9d894d398b7002af94aa3f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.130542 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8497edff1aaf45e48da16ad73a649c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.131335 13677 consensus_queue.cc:237] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.141712 13461 tablet_service.cc:1467] Processing CreateTablet for tablet fe0b468643254b1dba30573778f90be8 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240503 08:04:30.143061 13661 tablet_bootstrap.cc:654] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.143589 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe0b468643254b1dba30573778f90be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.132344 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13da943b8b4caa8da56b29cd48f93f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240503 08:04:30.148653 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6dbe71928443b09931a1b356044911" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:30.158252 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2325f0703f147ffabe6be77e2db6433" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:30.160503 13532 tablet_service.cc:1467] Processing CreateTablet for tablet fe0b468643254b1dba30573778f90be8 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240503 08:04:30.161885 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2325f0703f147ffabe6be77e2db6433" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:30.152062 13602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aae93c20681b4e4fa9b521edbb9e65e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 08:04:30.181589 13512 leader_election.cc:341] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:30.183249 13434 leader_election.cc:341] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.183498 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe0b468643254b1dba30573778f90be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.183719 13434 leader_election.cc:302] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:30.185304 13671 raft_consensus.cc:2726] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:30.195297 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13da943b8b4caa8da56b29cd48f93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.197712 13369 catalog_manager.cc:5484] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
W20240503 08:04:30.189721 13511 leader_election.cc:341] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.199582 13511 leader_election.cc:302] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:30.200443 13675 raft_consensus.cc:2726] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:30.190399 13527 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13da943b8b4caa8da56b29cd48f93f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240503 08:04:30.202097 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13da943b8b4caa8da56b29cd48f93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.209538 13661 tablet_bootstrap.cc:492] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.210034 13661 ts_tablet_manager.cc:1367] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.148s user 0.028s sys 0.004s
I20240503 08:04:30.212687 13661 raft_consensus.cc:348] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.213505 13661 raft_consensus.cc:374] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.213791 13661 raft_consensus.cc:729] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.214329 13453 tablet_service.cc:1467] Processing CreateTablet for tablet 878ffcf68a1b4726a683f6590d94704c (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240503 08:04:30.214617 13661 consensus_queue.cc:260] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.215685 13453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 878ffcf68a1b4726a683f6590d94704c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.218600 13661 ts_tablet_manager.cc:1398] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20240503 08:04:30.219594 13661 tablet_bootstrap.cc:492] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.220185 13457 tablet_service.cc:1467] Processing CreateTablet for tablet 691511776999428eae4eee8e73de08ce (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240503 08:04:30.221531 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691511776999428eae4eee8e73de08ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.234412 13660 tablet_bootstrap.cc:492] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:30.234951 13660 ts_tablet_manager.cc:1367] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.204s user 0.015s sys 0.012s
I20240503 08:04:30.235669 13605 tablet_service.cc:1467] Processing CreateTablet for tablet aa42fc3b9d894d398b7002af94aa3f40 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240503 08:04:30.236929 13671 raft_consensus.cc:482] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.237344 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa42fc3b9d894d398b7002af94aa3f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.237154 13528 tablet_service.cc:1467] Processing CreateTablet for tablet 6969c8edb68343339904e50c63bbdace (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240503 08:04:30.237428 13671 raft_consensus.cc:504] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.238653 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6969c8edb68343339904e50c63bbdace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.242995 13604 tablet_service.cc:1467] Processing CreateTablet for tablet c2e79e29310743ff852e4dfe1d8da13f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240503 08:04:30.244897 13604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e79e29310743ff852e4dfe1d8da13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.251256 13607 tablet_service.cc:1467] Processing CreateTablet for tablet 6969c8edb68343339904e50c63bbdace (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240503 08:04:30.252529 13603 tablet_service.cc:1467] Processing CreateTablet for tablet fe0b468643254b1dba30573778f90be8 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240503 08:04:30.263793 13452 tablet_service.cc:1467] Processing CreateTablet for tablet 008315f470c24029bd9271aad3c67a09 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240503 08:04:30.265617 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 008315f470c24029bd9271aad3c67a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.267202 13661 tablet_bootstrap.cc:654] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.268708 13665 tablet_bootstrap.cc:492] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:30.268312 13660 raft_consensus.cc:348] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.269145 13660 raft_consensus.cc:374] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.269217 13665 ts_tablet_manager.cc:1367] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.213s user 0.024s sys 0.024s
I20240503 08:04:30.269518 13660 raft_consensus.cc:729] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.270408 13660 consensus_queue.cc:260] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.272123 13665 raft_consensus.cc:348] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.272346 13460 tablet_service.cc:1467] Processing CreateTablet for tablet 6969c8edb68343339904e50c63bbdace (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240503 08:04:30.272897 13665 raft_consensus.cc:374] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.273257 13665 raft_consensus.cc:729] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.273655 13460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6969c8edb68343339904e50c63bbdace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.274852 13601 tablet_service.cc:1467] Processing CreateTablet for tablet 878ffcf68a1b4726a683f6590d94704c (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240503 08:04:30.276284 13459 tablet_service.cc:1467] Processing CreateTablet for tablet 41e689a2dc7941258f45f121e31c75bd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240503 08:04:30.276245 13665 consensus_queue.cc:260] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.277652 13459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e689a2dc7941258f45f121e31c75bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.279361 13530 tablet_service.cc:1467] Processing CreateTablet for tablet 691511776999428eae4eee8e73de08ce (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240503 08:04:30.291914 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691511776999428eae4eee8e73de08ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.282498 13665 ts_tablet_manager.cc:1398] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.013s user 0.006s sys 0.002s
I20240503 08:04:30.288041 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2325f0703f147ffabe6be77e2db6433" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:30.289755 13671 leader_election.cc:288] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.290325 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2325f0703f147ffabe6be77e2db6433" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:30.293509 13677 raft_consensus.cc:482] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.293843 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 878ffcf68a1b4726a683f6590d94704c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.224228 13533 tablet_service.cc:1467] Processing CreateTablet for tablet c2e79e29310743ff852e4dfe1d8da13f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240503 08:04:30.243844 13456 tablet_service.cc:1467] Processing CreateTablet for tablet c2e79e29310743ff852e4dfe1d8da13f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240503 08:04:30.301999 13546 raft_consensus.cc:2454] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:30.301978 13677 raft_consensus.cc:504] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.302567 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e79e29310743ff852e4dfe1d8da13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.303504 13434 leader_election.cc:302] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.304178 13677 leader_election.cc:288] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:30.302078 13620 raft_consensus.cc:2454] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:30.306293 13665 tablet_bootstrap.cc:492] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:30.306607 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f02eda12b9544888367d6206e3b41dd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:30.308388 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f02eda12b9544888367d6206e3b41dd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:30.309000 13585 leader_election.cc:341] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.309036 13546 raft_consensus.cc:2454] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:30.310320 13583 leader_election.cc:302] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:30.285701 13610 tablet_service.cc:1467] Processing CreateTablet for tablet 008315f470c24029bd9271aad3c67a09 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240503 08:04:30.313591 13677 raft_consensus.cc:2781] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:30.313741 13671 raft_consensus.cc:2781] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:30.314002 13677 raft_consensus.cc:482] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:30.314410 13671 raft_consensus.cc:482] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:30.314461 13677 raft_consensus.cc:3037] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.314986 13671 raft_consensus.cc:3037] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.323462 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6969c8edb68343339904e50c63bbdace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.323303 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 691511776999428eae4eee8e73de08ce (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240503 08:04:30.324949 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691511776999428eae4eee8e73de08ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.327574 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe0b468643254b1dba30573778f90be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.332557 13610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 008315f470c24029bd9271aad3c67a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.335436 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e79e29310743ff852e4dfe1d8da13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.336431 13660 ts_tablet_manager.cc:1398] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.101s user 0.000s sys 0.010s
I20240503 08:04:30.337527 13660 tablet_bootstrap.cc:492] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:30.345943 13531 tablet_service.cc:1467] Processing CreateTablet for tablet 878ffcf68a1b4726a683f6590d94704c (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240503 08:04:30.347327 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 878ffcf68a1b4726a683f6590d94704c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.378561 13671 raft_consensus.cc:504] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:30.370260 13434 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 66848.2us
I20240503 08:04:30.380724 13671 leader_election.cc:288] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.386406 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2325f0703f147ffabe6be77e2db6433" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:30.330914 13677 raft_consensus.cc:504] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.388654 13546 raft_consensus.cc:3037] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.392169 13455 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2966873f74257a9461592adbf7573 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240503 08:04:30.393018 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2325f0703f147ffabe6be77e2db6433" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:30.393668 13620 raft_consensus.cc:3037] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.397961 13528 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2966873f74257a9461592adbf7573 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240503 08:04:30.398947 13534 tablet_service.cc:1467] Processing CreateTablet for tablet f166d4169b16438fa99c721ab4c393ff (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240503 08:04:30.398838 13461 tablet_service.cc:1467] Processing CreateTablet for tablet 88e49c587b8a40e7a6eee4ee6566de1e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240503 08:04:30.399546 13453 tablet_service.cc:1467] Processing CreateTablet for tablet f166d4169b16438fa99c721ab4c393ff (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240503 08:04:30.412264 13534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f166d4169b16438fa99c721ab4c393ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.402341 13536 tablet_service.cc:1467] Processing CreateTablet for tablet 41e689a2dc7941258f45f121e31c75bd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240503 08:04:30.403218 13454 tablet_service.cc:1467] Processing CreateTablet for tablet 0ef1acee9ffb4310bde01caa87d35754 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240503 08:04:30.393664 13455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2966873f74257a9461592adbf7573. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.416457 13528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2966873f74257a9461592adbf7573. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.416806 13454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef1acee9ffb4310bde01caa87d35754. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.408010 13660 tablet_bootstrap.cc:654] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.419431 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e49c587b8a40e7a6eee4ee6566de1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.421171 13453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f166d4169b16438fa99c721ab4c393ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.421624 13458 tablet_service.cc:1467] Processing CreateTablet for tablet d4999c55f4034c3d9997841353a35afd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240503 08:04:30.422927 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4999c55f4034c3d9997841353a35afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.423635 13529 tablet_service.cc:1467] Processing CreateTablet for tablet 008315f470c24029bd9271aad3c67a09 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240503 08:04:30.410611 13527 tablet_service.cc:1467] Processing CreateTablet for tablet 88e49c587b8a40e7a6eee4ee6566de1e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240503 08:04:30.432003 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f02eda12b9544888367d6206e3b41dd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:30.433990 13608 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2966873f74257a9461592adbf7573 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240503 08:04:30.434403 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 41e689a2dc7941258f45f121e31c75bd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240503 08:04:30.442902 13608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2966873f74257a9461592adbf7573. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 08:04:30.443387 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:30.444530 13604 tablet_service.cc:1467] Processing CreateTablet for tablet f166d4169b16438fa99c721ab4c393ff (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20240503 08:04:30.435531 13585 leader_election.cc:341] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.449281 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e689a2dc7941258f45f121e31c75bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.458786 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e49c587b8a40e7a6eee4ee6566de1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.458990 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e689a2dc7941258f45f121e31c75bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.464057 13604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f166d4169b16438fa99c721ab4c393ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.466138 13620 raft_consensus.cc:2454] T a2325f0703f147ffabe6be77e2db6433 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:30.458813 13546 raft_consensus.cc:2454] T a2325f0703f147ffabe6be77e2db6433 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:30.467458 13437 leader_election.cc:302] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.469384 13606 tablet_service.cc:1467] Processing CreateTablet for tablet 88e49c587b8a40e7a6eee4ee6566de1e (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240503 08:04:30.470000 13689 raft_consensus.cc:2781] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:30.471412 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 008315f470c24029bd9271aad3c67a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.472113 13661 tablet_bootstrap.cc:492] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.495473 13661 ts_tablet_manager.cc:1367] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.276s user 0.017s sys 0.010s
I20240503 08:04:30.496006 13689 raft_consensus.cc:686] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:30.497066 13689 consensus_queue.cc:237] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.498239 13661 raft_consensus.cc:348] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.500871 13535 tablet_service.cc:1467] Processing CreateTablet for tablet d4999c55f4034c3d9997841353a35afd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240503 08:04:30.501422 13606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e49c587b8a40e7a6eee4ee6566de1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.498991 13661 raft_consensus.cc:374] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.502835 13661 raft_consensus.cc:729] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.503963 13661 consensus_queue.cc:260] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.505448 13671 raft_consensus.cc:482] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.506141 13661 ts_tablet_manager.cc:1398] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240503 08:04:30.505924 13671 raft_consensus.cc:504] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.480326 13677 leader_election.cc:288] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:30.480556 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f02eda12b9544888367d6206e3b41dd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:30.507004 13546 raft_consensus.cc:3037] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.507959 13671 leader_election.cc:288] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.489643 13665 tablet_bootstrap.cc:654] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.493468 13532 tablet_service.cc:1467] Processing CreateTablet for tablet 0ef1acee9ffb4310bde01caa87d35754 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240503 08:04:30.493958 13602 tablet_service.cc:1467] Processing CreateTablet for tablet 0ef1acee9ffb4310bde01caa87d35754 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240503 08:04:30.454612 13460 tablet_service.cc:1467] Processing CreateTablet for tablet 1daae5df6ae6403e895eefcf5fe05144 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240503 08:04:30.502182 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4999c55f4034c3d9997841353a35afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.519203 13661 tablet_bootstrap.cc:492] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.522187 13531 tablet_service.cc:1467] Processing CreateTablet for tablet 13577f32ef6f412fb64c5ae26ec2f13b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240503 08:04:30.525425 13460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1daae5df6ae6403e895eefcf5fe05144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.525280 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "911fa6daf5354732a1c3741fe6da0fa7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:30.528968 13546 raft_consensus.cc:2454] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:30.530133 13527 tablet_service.cc:1467] Processing CreateTablet for tablet 556938a7848d410b8ee50c324d57228f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240503 08:04:30.532923 13456 tablet_service.cc:1467] Processing CreateTablet for tablet 556938a7848d410b8ee50c324d57228f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240503 08:04:30.534155 13530 tablet_service.cc:1467] Processing CreateTablet for tablet 1daae5df6ae6403e895eefcf5fe05144 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240503 08:04:30.547544 13601 tablet_service.cc:1467] Processing CreateTablet for tablet 13577f32ef6f412fb64c5ae26ec2f13b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240503 08:04:30.549216 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef1acee9ffb4310bde01caa87d35754. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.551666 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "911fa6daf5354732a1c3741fe6da0fa7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:30.552256 13531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13577f32ef6f412fb64c5ae26ec2f13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.552356 13620 raft_consensus.cc:2454] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:30.553767 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556938a7848d410b8ee50c324d57228f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.544680 13545 raft_consensus.cc:2454] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:30.546314 13604 tablet_service.cc:1467] Processing CreateTablet for tablet 1daae5df6ae6403e895eefcf5fe05144 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240503 08:04:30.553608 13437 leader_election.cc:302] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.556092 13671 raft_consensus.cc:2781] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:30.556511 13671 raft_consensus.cc:482] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:30.556854 13671 raft_consensus.cc:3037] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.537113 13452 tablet_service.cc:1467] Processing CreateTablet for tablet 13577f32ef6f412fb64c5ae26ec2f13b (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240503 08:04:30.571856 13610 tablet_service.cc:1467] Processing CreateTablet for tablet d4999c55f4034c3d9997841353a35afd (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240503 08:04:30.550837 13607 tablet_service.cc:1467] Processing CreateTablet for tablet 556938a7848d410b8ee50c324d57228f (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240503 08:04:30.577953 13583 leader_election.cc:302] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:30.579185 13677 raft_consensus.cc:2781] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:30.579712 13677 raft_consensus.cc:686] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:30.556070 13609 tablet_service.cc:1467] Processing CreateTablet for tablet b8c70e75f946416cb5c23f68cfe59b98 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240503 08:04:30.580550 13677 consensus_queue.cc:237] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.574939 13661 tablet_bootstrap.cc:654] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.567497 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13577f32ef6f412fb64c5ae26ec2f13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.534818 13533 tablet_service.cc:1467] Processing CreateTablet for tablet b8c70e75f946416cb5c23f68cfe59b98 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240503 08:04:30.583798 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1daae5df6ae6403e895eefcf5fe05144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.595109 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556938a7848d410b8ee50c324d57228f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.608264 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13577f32ef6f412fb64c5ae26ec2f13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.611477 13610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4999c55f4034c3d9997841353a35afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.612166 13602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef1acee9ffb4310bde01caa87d35754. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.627478 13604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1daae5df6ae6403e895eefcf5fe05144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.627622 13457 tablet_service.cc:1467] Processing CreateTablet for tablet b8c70e75f946416cb5c23f68cfe59b98 (DEFAULT_TABLE table=test-workload [id=1898f06dfb474368bb2b92a9245afac3]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240503 08:04:30.628484 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556938a7848d410b8ee50c324d57228f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.628894 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c70e75f946416cb5c23f68cfe59b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.643754 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c70e75f946416cb5c23f68cfe59b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.651878 13665 tablet_bootstrap.cc:492] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:30.652424 13665 ts_tablet_manager.cc:1367] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.346s user 0.017s sys 0.003s
I20240503 08:04:30.652483 13660 tablet_bootstrap.cc:492] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:30.653048 13660 ts_tablet_manager.cc:1367] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.316s user 0.019s sys 0.010s
I20240503 08:04:30.655839 13660 raft_consensus.cc:348] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.656580 13660 raft_consensus.cc:374] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.656877 13660 raft_consensus.cc:729] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.657655 13660 consensus_queue.cc:260] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.661039 13660 ts_tablet_manager.cc:1398] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240503 08:04:30.662015 13660 tablet_bootstrap.cc:492] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:30.671818 13677 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:30.680722 13661 tablet_bootstrap.cc:492] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.681305 13661 ts_tablet_manager.cc:1367] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.162s user 0.013s sys 0.000s
I20240503 08:04:30.672056 13367 catalog_manager.cc:5484] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:30.687549 13661 raft_consensus.cc:348] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.688324 13661 raft_consensus.cc:374] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.688618 13661 raft_consensus.cc:729] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.689347 13661 consensus_queue.cc:260] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.691318 13661 ts_tablet_manager.cc:1398] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20240503 08:04:30.692163 13661 tablet_bootstrap.cc:492] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.695940 13369 catalog_manager.cc:5484] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:30.698385 13661 tablet_bootstrap.cc:654] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.663471 13671 raft_consensus.cc:504] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.701041 13671 leader_election.cc:288] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.702215 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "911fa6daf5354732a1c3741fe6da0fa7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:30.702850 13620 raft_consensus.cc:3037] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.703173 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "911fa6daf5354732a1c3741fe6da0fa7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:30.703878 13546 raft_consensus.cc:3037] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.704128 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c70e75f946416cb5c23f68cfe59b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:30.716684 13620 raft_consensus.cc:2454] T 911fa6daf5354732a1c3741fe6da0fa7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:30.718014 13437 leader_election.cc:302] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.718888 13671 raft_consensus.cc:2781] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:30.719611 13671 raft_consensus.cc:686] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:30.720402 13671 consensus_queue.cc:237] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.722679 13546 raft_consensus.cc:2454] T 911fa6daf5354732a1c3741fe6da0fa7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:30.725189 13665 raft_consensus.cc:348] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.725961 13665 raft_consensus.cc:374] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.727304 13665 raft_consensus.cc:729] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.728015 13665 consensus_queue.cc:260] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.743816 13661 tablet_bootstrap.cc:492] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.744328 13661 ts_tablet_manager.cc:1367] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.052s user 0.005s sys 0.015s
I20240503 08:04:30.746532 13665 ts_tablet_manager.cc:1398] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.094s user 0.007s sys 0.001s
I20240503 08:04:30.755168 13367 catalog_manager.cc:5484] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:30.770093 13671 raft_consensus.cc:482] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.770656 13671 raft_consensus.cc:504] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:30.772910 13583 consensus_peers.cc:485] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:30.774230 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b181d58f5aef42679e79edcd3b81c20b" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:30.774909 13620 raft_consensus.cc:2454] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:30.775856 13660 tablet_bootstrap.cc:654] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.776051 13437 leader_election.cc:302] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.777559 13689 raft_consensus.cc:2781] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:30.777906 13689 raft_consensus.cc:482] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:30.778244 13689 raft_consensus.cc:3037] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.779970 13661 raft_consensus.cc:348] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.780720 13661 raft_consensus.cc:374] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.781086 13661 raft_consensus.cc:729] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.774241 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b181d58f5aef42679e79edcd3b81c20b" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:30.785264 13434 leader_election.cc:341] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.776870 13665 tablet_bootstrap.cc:492] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:30.785892 13671 leader_election.cc:288] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.787945 13689 raft_consensus.cc:504] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.797427 13689 leader_election.cc:288] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.798370 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b181d58f5aef42679e79edcd3b81c20b" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:30.798370 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b181d58f5aef42679e79edcd3b81c20b" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:30.799729 13620 raft_consensus.cc:3037] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:30.800030 13434 leader_election.cc:341] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.791234 13661 consensus_queue.cc:260] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.817979 13661 ts_tablet_manager.cc:1398] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.073s user 0.001s sys 0.005s
I20240503 08:04:30.823412 13661 tablet_bootstrap.cc:492] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.824565 13620 raft_consensus.cc:2454] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:30.825628 13437 leader_election.cc:302] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:30.826246 13660 tablet_bootstrap.cc:492] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:30.826782 13660 ts_tablet_manager.cc:1367] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.165s user 0.016s sys 0.026s
I20240503 08:04:30.826756 13689 raft_consensus.cc:2781] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:30.827476 13689 raft_consensus.cc:686] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:30.828677 13689 consensus_queue.cc:237] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.830046 13661 tablet_bootstrap.cc:654] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.832546 13665 tablet_bootstrap.cc:654] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.833523 13660 raft_consensus.cc:348] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.834081 13660 raft_consensus.cc:374] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.834450 13660 raft_consensus.cc:729] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.835392 13660 consensus_queue.cc:260] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.837618 13665 tablet_bootstrap.cc:492] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:30.838110 13665 ts_tablet_manager.cc:1367] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.061s user 0.011s sys 0.000s
I20240503 08:04:30.838409 13660 ts_tablet_manager.cc:1398] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240503 08:04:30.839841 13660 tablet_bootstrap.cc:492] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:30.842109 13665 raft_consensus.cc:348] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.842681 13665 raft_consensus.cc:374] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.843040 13665 raft_consensus.cc:729] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
W20240503 08:04:30.843679 13497 tablet.cc:2369] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 08:04:30.843853 13665 consensus_queue.cc:260] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.846194 13660 tablet_bootstrap.cc:654] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.846289 13665 ts_tablet_manager.cc:1398] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.002s sys 0.005s
I20240503 08:04:30.847384 13665 tablet_bootstrap.cc:492] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:30.842617 13369 catalog_manager.cc:5484] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:30.856271 13665 tablet_bootstrap.cc:654] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.868530 13665 tablet_bootstrap.cc:492] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:30.869171 13665 ts_tablet_manager.cc:1367] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.009s
I20240503 08:04:30.872247 13665 raft_consensus.cc:348] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.872987 13665 raft_consensus.cc:374] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.873140 13661 tablet_bootstrap.cc:492] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.873327 13665 raft_consensus.cc:729] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.873790 13661 ts_tablet_manager.cc:1367] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.015s
I20240503 08:04:30.874366 13665 consensus_queue.cc:260] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.876698 13661 raft_consensus.cc:348] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.877209 13665 ts_tablet_manager.cc:1398] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.006s sys 0.003s
I20240503 08:04:30.877650 13661 raft_consensus.cc:374] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.877970 13661 raft_consensus.cc:729] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.879199 13665 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:30.879621 13661 consensus_queue.cc:260] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.881659 13660 tablet_bootstrap.cc:492] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:30.881994 13661 ts_tablet_manager.cc:1398] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.008s user 0.000s sys 0.004s
I20240503 08:04:30.882165 13660 ts_tablet_manager.cc:1367] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.043s user 0.018s sys 0.018s
I20240503 08:04:30.885776 13675 raft_consensus.cc:482] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.886298 13675 raft_consensus.cc:504] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.888304 13675 leader_election.cc:288] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.889125 13660 raft_consensus.cc:348] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.889884 13660 raft_consensus.cc:374] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.889919 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6dbe71928443b09931a1b356044911" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:30.890298 13660 raft_consensus.cc:729] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.890640 13470 raft_consensus.cc:2454] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:30.891101 13660 consensus_queue.cc:260] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.891934 13511 leader_election.cc:302] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.882979 13661 tablet_bootstrap.cc:492] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.883188 13704 consensus_queue.cc:1035] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:30.893899 13660 ts_tablet_manager.cc:1398] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.001s sys 0.005s
I20240503 08:04:30.893765 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6dbe71928443b09931a1b356044911" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:30.894331 13675 raft_consensus.cc:2781] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:30.894562 13620 raft_consensus.cc:2454] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:30.894714 13675 raft_consensus.cc:482] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:30.895051 13675 raft_consensus.cc:3037] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.901087 13661 tablet_bootstrap.cc:654] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.902248 13665 tablet_bootstrap.cc:654] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.903218 13660 tablet_bootstrap.cc:492] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:30.908870 13675 raft_consensus.cc:504] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.913179 13675 leader_election.cc:288] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:30.914067 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6dbe71928443b09931a1b356044911" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:30.914829 13470 raft_consensus.cc:3037] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.919718 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6dbe71928443b09931a1b356044911" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:30.920617 13620 raft_consensus.cc:3037] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:30.922574 13661 tablet_bootstrap.cc:492] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.923082 13661 ts_tablet_manager.cc:1367] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.040s user 0.018s sys 0.007s
I20240503 08:04:30.925308 13470 raft_consensus.cc:2454] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:30.925935 13661 raft_consensus.cc:348] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.926772 13661 raft_consensus.cc:374] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.927043 13661 raft_consensus.cc:729] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.926959 13511 leader_election.cc:302] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:30.928602 13710 raft_consensus.cc:2781] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:30.928275 13661 consensus_queue.cc:260] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.929302 13710 raft_consensus.cc:686] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:30.930441 13661 ts_tablet_manager.cc:1398] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240503 08:04:30.931602 13661 tablet_bootstrap.cc:492] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.931207 13710 consensus_queue.cc:237] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } }
I20240503 08:04:30.936621 13661 tablet_bootstrap.cc:654] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.940881 13620 raft_consensus.cc:2454] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:30.954406 13660 tablet_bootstrap.cc:654] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.954936 13665 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:30.955454 13665 ts_tablet_manager.cc:1367] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.077s user 0.031s sys 0.017s
I20240503 08:04:30.958202 13665 raft_consensus.cc:348] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.958717 13677 raft_consensus.cc:482] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:30.959010 13665 raft_consensus.cc:374] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.959466 13665 raft_consensus.cc:729] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.959270 13677 raft_consensus.cc:504] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.960207 13665 consensus_queue.cc:260] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.961297 13677 leader_election.cc:288] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:30.962168 13671 consensus_queue.cc:1035] T a2325f0703f147ffabe6be77e2db6433 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:30.962797 13665 ts_tablet_manager.cc:1398] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240503 08:04:30.963358 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6234f05657b4c0fbb21339b1a63d571" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
W20240503 08:04:30.964838 13585 leader_election.cc:341] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.967253 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6234f05657b4c0fbb21339b1a63d571" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:30.967965 13665 tablet_bootstrap.cc:492] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
W20240503 08:04:30.968839 13583 leader_election.cc:341] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:30.969290 13583 leader_election.cc:302] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:30.970669 13677 raft_consensus.cc:2726] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:30.973320 13661 tablet_bootstrap.cc:492] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:30.973803 13661 ts_tablet_manager.cc:1367] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.042s user 0.018s sys 0.004s
I20240503 08:04:30.972360 13366 catalog_manager.cc:5484] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:30.976428 13661 raft_consensus.cc:348] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.977171 13661 raft_consensus.cc:374] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.977473 13661 raft_consensus.cc:729] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.978042 13661 consensus_queue.cc:260] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.979933 13661 ts_tablet_manager.cc:1398] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:30.982587 13665 tablet_bootstrap.cc:654] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.983575 13660 tablet_bootstrap.cc:492] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:30.984086 13660 ts_tablet_manager.cc:1367] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.081s user 0.006s sys 0.007s
I20240503 08:04:30.985265 13661 tablet_bootstrap.cc:492] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:30.987432 13660 raft_consensus.cc:348] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.988384 13660 raft_consensus.cc:374] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:30.988706 13660 raft_consensus.cc:729] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:30.989713 13660 consensus_queue.cc:260] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:30.991336 13661 tablet_bootstrap.cc:654] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:30.998843 13677 consensus_queue.cc:1035] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:31.006296 13710 raft_consensus.cc:482] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.007006 13710 raft_consensus.cc:504] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.009457 13710 leader_election.cc:288] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.011328 13660 ts_tablet_manager.cc:1398] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20240503 08:04:31.011582 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fd7638c4454552a7aeb0aa85bef63b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.012310 13660 tablet_bootstrap.cc:492] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.012527 13470 raft_consensus.cc:2454] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:31.017752 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fd7638c4454552a7aeb0aa85bef63b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.018546 13619 raft_consensus.cc:2454] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:31.023731 13660 tablet_bootstrap.cc:654] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.014736 13511 leader_election.cc:302] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.031070 13470 raft_consensus.cc:3037] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.031657 13675 raft_consensus.cc:2781] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.032753 13675 raft_consensus.cc:482] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.033109 13675 raft_consensus.cc:3037] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.040925 13661 tablet_bootstrap.cc:492] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.042181 13675 raft_consensus.cc:504] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.043362 13661 ts_tablet_manager.cc:1367] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.058s user 0.012s sys 0.026s
I20240503 08:04:31.044675 13675 leader_election.cc:288] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.045446 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fd7638c4454552a7aeb0aa85bef63b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:31.046046 13471 raft_consensus.cc:3037] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.047534 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fd7638c4454552a7aeb0aa85bef63b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.048206 13620 raft_consensus.cc:3037] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.048380 13661 raft_consensus.cc:348] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.049118 13661 raft_consensus.cc:374] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.049397 13661 raft_consensus.cc:729] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.050725 13661 consensus_queue.cc:260] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.053865 13471 raft_consensus.cc:2454] T 07fd7638c4454552a7aeb0aa85bef63b P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:31.055305 13620 raft_consensus.cc:2454] T 07fd7638c4454552a7aeb0aa85bef63b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:31.055974 13511 leader_election.cc:302] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.057724 13675 raft_consensus.cc:2781] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.061023 13671 raft_consensus.cc:482] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.059206 13661 ts_tablet_manager.cc:1398] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20240503 08:04:31.061484 13671 raft_consensus.cc:504] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.062433 13661 tablet_bootstrap.cc:492] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.062199 13680 consensus_queue.cc:1035] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:31.065570 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f02eda12b9544888367d6206e3b41dd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.059710 13665 tablet_bootstrap.cc:492] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.067421 13671 leader_election.cc:288] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.067679 13675 raft_consensus.cc:686] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:31.069250 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f02eda12b9544888367d6206e3b41dd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.069160 13675 consensus_queue.cc:237] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.070842 13437 leader_election.cc:302] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:31.067466 13665 ts_tablet_manager.cc:1367] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.100s user 0.024s sys 0.036s
I20240503 08:04:31.073117 13660 tablet_bootstrap.cc:492] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.073582 13660 ts_tablet_manager.cc:1367] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.061s user 0.010s sys 0.003s
I20240503 08:04:31.074934 13665 raft_consensus.cc:348] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.075958 13665 raft_consensus.cc:374] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.075920 13660 raft_consensus.cc:348] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.076403 13671 raft_consensus.cc:2726] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 08:04:31.076406 13665 raft_consensus.cc:729] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.076851 13660 raft_consensus.cc:374] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.077445 13660 raft_consensus.cc:729] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.077570 13665 consensus_queue.cc:260] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.078617 13661 tablet_bootstrap.cc:654] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.078251 13660 consensus_queue.cc:260] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.080144 13660 ts_tablet_manager.cc:1398] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20240503 08:04:31.081027 13660 tablet_bootstrap.cc:492] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.092830 13665 ts_tablet_manager.cc:1398] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20240503 08:04:31.093799 13665 tablet_bootstrap.cc:492] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.094673 13710 raft_consensus.cc:482] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.095332 13710 raft_consensus.cc:504] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.100117 13660 tablet_bootstrap.cc:654] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.100412 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.097604 13710 leader_election.cc:288] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.103595 13661 tablet_bootstrap.cc:492] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.104413 13661 ts_tablet_manager.cc:1367] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.042s user 0.017s sys 0.000s
I20240503 08:04:31.107043 13661 raft_consensus.cc:348] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.107844 13661 raft_consensus.cc:374] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.108127 13661 raft_consensus.cc:729] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.108064 13671 consensus_queue.cc:1035] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:31.108944 13661 consensus_queue.cc:260] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.110570 13665 tablet_bootstrap.cc:654] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.116604 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768f7ed6c5cc40bd947c30cf5450c8f1" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.118687 13661 ts_tablet_manager.cc:1398] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20240503 08:04:31.119941 13661 tablet_bootstrap.cc:492] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.119273 13512 leader_election.cc:302] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.118654 13367 catalog_manager.cc:5484] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:31.120980 13675 raft_consensus.cc:2726] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.126109 13661 tablet_bootstrap.cc:654] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.136989 13665 tablet_bootstrap.cc:492] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.137792 13665 ts_tablet_manager.cc:1367] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.044s user 0.015s sys 0.003s
I20240503 08:04:31.144431 13660 tablet_bootstrap.cc:492] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.144914 13660 ts_tablet_manager.cc:1367] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.064s user 0.017s sys 0.031s
I20240503 08:04:31.146782 13665 raft_consensus.cc:348] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.147089 13704 consensus_queue.cc:1035] T 911fa6daf5354732a1c3741fe6da0fa7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:31.147630 13665 raft_consensus.cc:374] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.148097 13665 raft_consensus.cc:729] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.148429 13660 raft_consensus.cc:348] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.149124 13660 raft_consensus.cc:374] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.149407 13660 raft_consensus.cc:729] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.148910 13665 consensus_queue.cc:260] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.150395 13660 consensus_queue.cc:260] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.152087 13665 ts_tablet_manager.cc:1398] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20240503 08:04:31.153075 13665 tablet_bootstrap.cc:492] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.153581 13660 ts_tablet_manager.cc:1398] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:31.154425 13660 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.158524 13665 tablet_bootstrap.cc:654] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.159466 13660 tablet_bootstrap.cc:654] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.161253 13661 tablet_bootstrap.cc:492] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.161734 13661 ts_tablet_manager.cc:1367] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.042s user 0.013s sys 0.005s
I20240503 08:04:31.167996 13661 raft_consensus.cc:348] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.168794 13661 raft_consensus.cc:374] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.169085 13661 raft_consensus.cc:729] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.169723 13661 consensus_queue.cc:260] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.170548 13660 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.171617 13660 ts_tablet_manager.cc:1367] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20240503 08:04:31.174906 13660 raft_consensus.cc:348] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.175649 13660 raft_consensus.cc:374] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.175954 13660 raft_consensus.cc:729] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.176143 13661 ts_tablet_manager.cc:1398] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20240503 08:04:31.177217 13661 tablet_bootstrap.cc:492] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.177488 13660 consensus_queue.cc:260] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.177942 13710 raft_consensus.cc:482] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.178442 13710 raft_consensus.cc:504] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.182082 13665 tablet_bootstrap.cc:492] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.182291 13660 ts_tablet_manager.cc:1398] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20240503 08:04:31.182560 13665 ts_tablet_manager.cc:1367] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.012s
I20240503 08:04:31.183118 13660 tablet_bootstrap.cc:492] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.183471 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b181d58f5aef42679e79edcd3b81c20b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.183557 13710 leader_election.cc:288] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.183161 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b181d58f5aef42679e79edcd3b81c20b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.189617 13620 raft_consensus.cc:2379] T b181d58f5aef42679e79edcd3b81c20b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:31.184940 13665 raft_consensus.cc:348] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.191008 13665 raft_consensus.cc:374] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.190738 13512 leader_election.cc:302] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.191324 13665 raft_consensus.cc:729] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.191988 13665 consensus_queue.cc:260] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.193107 13710 raft_consensus.cc:3037] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.193573 13665 ts_tablet_manager.cc:1398] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20240503 08:04:31.190781 13661 tablet_bootstrap.cc:654] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.194435 13665 tablet_bootstrap.cc:492] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.194458 13660 tablet_bootstrap.cc:654] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.196849 13546 raft_consensus.cc:3037] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.204944 13710 raft_consensus.cc:2726] T b181d58f5aef42679e79edcd3b81c20b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.216130 13660 tablet_bootstrap.cc:492] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.216667 13661 tablet_bootstrap.cc:492] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.216830 13660 ts_tablet_manager.cc:1367] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.007s
I20240503 08:04:31.217154 13661 ts_tablet_manager.cc:1367] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.008s
I20240503 08:04:31.218269 13665 tablet_bootstrap.cc:654] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.219502 13660 raft_consensus.cc:348] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.219642 13661 raft_consensus.cc:348] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.220336 13660 raft_consensus.cc:374] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.220453 13661 raft_consensus.cc:374] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.220890 13660 raft_consensus.cc:729] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.220957 13661 raft_consensus.cc:729] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.221741 13661 consensus_queue.cc:260] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.223946 13661 ts_tablet_manager.cc:1398] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240503 08:04:31.221741 13660 consensus_queue.cc:260] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.225558 13660 ts_tablet_manager.cc:1398] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240503 08:04:31.226480 13660 tablet_bootstrap.cc:492] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.227458 13661 tablet_bootstrap.cc:492] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.231705 13660 tablet_bootstrap.cc:654] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.233278 13661 tablet_bootstrap.cc:654] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.255806 13704 consensus_queue.cc:1035] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:31.261757 13665 tablet_bootstrap.cc:492] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.262317 13665 ts_tablet_manager.cc:1367] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.068s user 0.035s sys 0.019s
I20240503 08:04:31.264647 13665 raft_consensus.cc:348] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.265313 13665 raft_consensus.cc:374] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.265427 13661 tablet_bootstrap.cc:492] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.265604 13665 raft_consensus.cc:729] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.265887 13661 ts_tablet_manager.cc:1367] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.013s
I20240503 08:04:31.266341 13665 consensus_queue.cc:260] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.267633 13689 consensus_queue.cc:1035] T b181d58f5aef42679e79edcd3b81c20b P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:31.269398 13660 tablet_bootstrap.cc:492] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.269605 13665 ts_tablet_manager.cc:1398] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20240503 08:04:31.269829 13660 ts_tablet_manager.cc:1367] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.044s user 0.032s sys 0.002s
I20240503 08:04:31.270443 13665 tablet_bootstrap.cc:492] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.270100 13661 raft_consensus.cc:348] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.270771 13661 raft_consensus.cc:374] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.271023 13661 raft_consensus.cc:729] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.271734 13661 consensus_queue.cc:260] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.273698 13661 ts_tablet_manager.cc:1398] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20240503 08:04:31.273362 13660 raft_consensus.cc:348] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.274120 13660 raft_consensus.cc:374] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.274379 13660 raft_consensus.cc:729] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.274968 13660 consensus_queue.cc:260] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.276122 13665 tablet_bootstrap.cc:654] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.277577 13660 ts_tablet_manager.cc:1398] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20240503 08:04:31.277855 13661 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.278409 13660 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.283411 13661 tablet_bootstrap.cc:654] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.285547 13660 tablet_bootstrap.cc:654] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.299185 13689 raft_consensus.cc:482] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.300017 13689 raft_consensus.cc:504] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.302304 13689 leader_election.cc:288] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.304415 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ca0d7dd59d4881b4c9e6a2a6a8a505" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.308013 13546 raft_consensus.cc:2454] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:31.309044 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ca0d7dd59d4881b4c9e6a2a6a8a505" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.309294 13434 leader_election.cc:302] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
W20240503 08:04:31.310936 13437 leader_election.cc:341] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.310375 13689 raft_consensus.cc:2781] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.311554 13689 raft_consensus.cc:482] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.311904 13689 raft_consensus.cc:3037] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.314123 13660 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.314620 13660 ts_tablet_manager.cc:1367] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.036s user 0.009s sys 0.007s
I20240503 08:04:31.328280 13661 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.328845 13661 ts_tablet_manager.cc:1367] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.015s
I20240503 08:04:31.317791 13660 raft_consensus.cc:348] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.330817 13660 raft_consensus.cc:374] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.331099 13660 raft_consensus.cc:729] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.331300 13661 raft_consensus.cc:348] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.331983 13661 raft_consensus.cc:374] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.332233 13661 raft_consensus.cc:729] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.331826 13660 consensus_queue.cc:260] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.323011 13689 raft_consensus.cc:504] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.333729 13660 ts_tablet_manager.cc:1398] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.019s user 0.002s sys 0.003s
I20240503 08:04:31.336547 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ca0d7dd59d4881b4c9e6a2a6a8a505" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:31.337209 13546 raft_consensus.cc:3037] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.337227 13704 raft_consensus.cc:482] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.338155 13704 raft_consensus.cc:504] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.344494 13704 leader_election.cc:288] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.345643 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1a896bcf18443fb65ff2a59c8dfe3f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:31.347401 13434 leader_election.cc:341] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.348877 13689 leader_election.cc:288] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.339486 13661 consensus_queue.cc:260] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.350613 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ca0d7dd59d4881b4c9e6a2a6a8a505" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.351316 13661 ts_tablet_manager.cc:1398] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.022s user 0.008s sys 0.000s
W20240503 08:04:31.352088 13437 leader_election.cc:341] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.354044 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1a896bcf18443fb65ff2a59c8dfe3f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:31.355401 13437 leader_election.cc:341] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.355727 13546 raft_consensus.cc:2454] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:31.355870 13437 leader_election.cc:302] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:31.356750 13689 raft_consensus.cc:2726] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.356833 13434 leader_election.cc:302] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:31.357898 13661 tablet_bootstrap.cc:492] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.358544 13689 raft_consensus.cc:2781] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.359341 13689 raft_consensus.cc:686] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:31.360272 13689 consensus_queue.cc:237] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.361819 13660 tablet_bootstrap.cc:492] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.368796 13665 tablet_bootstrap.cc:492] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.368949 13660 tablet_bootstrap.cc:654] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.368747 13704 raft_consensus.cc:482] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.369479 13704 raft_consensus.cc:504] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.370407 13661 tablet_bootstrap.cc:654] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.371544 13704 leader_election.cc:288] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.369467 13665 ts_tablet_manager.cc:1367] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.099s user 0.032s sys 0.027s
I20240503 08:04:31.372563 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6f8cbdf9a94be9a490fc84cf1c3d7e" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:31.374020 13434 leader_election.cc:341] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.374519 13665 raft_consensus.cc:348] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.374990 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6f8cbdf9a94be9a490fc84cf1c3d7e" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.375335 13665 raft_consensus.cc:374] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.375636 13665 raft_consensus.cc:729] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.376262 13665 consensus_queue.cc:260] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.375644 13620 raft_consensus.cc:2454] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:31.380244 13665 ts_tablet_manager.cc:1398] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240503 08:04:31.381914 13665 tablet_bootstrap.cc:492] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.385380 13437 leader_election.cc:302] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:31.386080 13689 raft_consensus.cc:2781] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.386499 13689 raft_consensus.cc:482] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.386921 13689 raft_consensus.cc:3037] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.392019 13661 tablet_bootstrap.cc:492] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.393146 13665 tablet_bootstrap.cc:654] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.392966 13369 catalog_manager.cc:5484] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:31.393858 13660 tablet_bootstrap.cc:492] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.397567 13661 ts_tablet_manager.cc:1367] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.005s
I20240503 08:04:31.398494 13704 raft_consensus.cc:482] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.398969 13704 raft_consensus.cc:504] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.400935 13704 leader_election.cc:288] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.397615 13660 ts_tablet_manager.cc:1367] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.048s user 0.014s sys 0.011s
I20240503 08:04:31.402083 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2dd11388594890a81f290c0bac6840" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.402608 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2dd11388594890a81f290c0bac6840" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.403115 13546 raft_consensus.cc:2454] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
W20240503 08:04:31.404297 13437 leader_election.cc:341] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.404922 13704 raft_consensus.cc:482] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.404862 13434 leader_election.cc:302] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:31.405542 13704 raft_consensus.cc:504] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.406512 13671 raft_consensus.cc:2781] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.407016 13671 raft_consensus.cc:482] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.407603 13671 raft_consensus.cc:3037] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.408048 13704 leader_election.cc:288] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.408748 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e609185a44d14a5473954db083b1a" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.409405 13620 raft_consensus.cc:2454] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:31.410261 13660 raft_consensus.cc:348] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.410960 13660 raft_consensus.cc:374] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.410842 13437 leader_election.cc:302] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.411273 13660 raft_consensus.cc:729] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.411850 13704 raft_consensus.cc:2781] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.412321 13704 raft_consensus.cc:482] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.412710 13704 raft_consensus.cc:3037] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.413786 13661 raft_consensus.cc:348] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.415158 13661 raft_consensus.cc:374] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.415436 13661 raft_consensus.cc:729] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.416097 13661 consensus_queue.cc:260] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.418684 13710 consensus_queue.cc:1035] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:31.411943 13660 consensus_queue.cc:260] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.423409 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e609185a44d14a5473954db083b1a" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:31.424768 13434 leader_election.cc:341] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.429773 13704 raft_consensus.cc:504] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.443521 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e609185a44d14a5473954db083b1a" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.444389 13619 raft_consensus.cc:3037] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.444414 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e609185a44d14a5473954db083b1a" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
W20240503 08:04:31.446302 13434 leader_election.cc:341] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.448532 13710 raft_consensus.cc:482] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.449025 13710 raft_consensus.cc:504] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.452018 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6234f05657b4c0fbb21339b1a63d571" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.452665 13471 raft_consensus.cc:2454] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:31.453864 13511 leader_election.cc:302] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.454700 13675 raft_consensus.cc:2781] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.455029 13675 raft_consensus.cc:482] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.455365 13675 raft_consensus.cc:3037] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:31.457548 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:31.459908 13710 leader_election.cc:288] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.461272 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6234f05657b4c0fbb21339b1a63d571" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:31.462539 13619 raft_consensus.cc:2454] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:31.437270 13660 ts_tablet_manager.cc:1398] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20240503 08:04:31.434442 13689 raft_consensus.cc:504] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.462610 13618 raft_consensus.cc:2454] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:31.463994 13660 tablet_bootstrap.cc:492] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.464712 13689 leader_election.cc:288] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.465581 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6f8cbdf9a94be9a490fc84cf1c3d7e" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:31.466128 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6f8cbdf9a94be9a490fc84cf1c3d7e" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.466871 13618 raft_consensus.cc:3037] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:31.466975 13434 leader_election.cc:341] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.463111 13675 raft_consensus.cc:504] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.469955 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6234f05657b4c0fbb21339b1a63d571" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:31.470522 13471 raft_consensus.cc:3037] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.477062 13471 raft_consensus.cc:2454] T a6234f05657b4c0fbb21339b1a63d571 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:31.478235 13511 leader_election.cc:302] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.437258 13671 raft_consensus.cc:504] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.479466 13710 raft_consensus.cc:2781] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.479372 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6234f05657b4c0fbb21339b1a63d571" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.480067 13617 raft_consensus.cc:3037] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.481698 13618 raft_consensus.cc:2454] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:31.483414 13437 leader_election.cc:302] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:31.488276 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2dd11388594890a81f290c0bac6840" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:31.489082 13546 raft_consensus.cc:3037] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.489730 13616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2dd11388594890a81f290c0bac6840" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.490270 13437 leader_election.cc:302] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:31.491253 13749 raft_consensus.cc:2781] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
W20240503 08:04:31.491554 13437 leader_election.cc:341] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.491755 13749 raft_consensus.cc:686] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:31.492758 13749 consensus_queue.cc:237] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.495276 13689 raft_consensus.cc:2781] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.497830 13546 raft_consensus.cc:2454] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:31.499303 13434 leader_election.cc:302] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:31.500288 13671 leader_election.cc:288] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.502074 13704 leader_election.cc:288] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.500897 13675 leader_election.cc:288] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.515584 13677 raft_consensus.cc:482] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.517109 13677 raft_consensus.cc:504] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.519966 13677 leader_election.cc:288] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.522240 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1b40442f084e05b58edb1d96de3fa3" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.523051 13660 tablet_bootstrap.cc:654] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.516252 13671 raft_consensus.cc:2781] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.526598 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1b40442f084e05b58edb1d96de3fa3" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.527192 13665 tablet_bootstrap.cc:492] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.527617 13665 ts_tablet_manager.cc:1367] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.146s user 0.021s sys 0.028s
W20240503 08:04:31.528103 13585 leader_election.cc:341] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.525962 13546 raft_consensus.cc:2454] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:31.529573 13583 leader_election.cc:302] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.530802 13677 raft_consensus.cc:2781] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.531186 13677 raft_consensus.cc:482] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.531533 13677 raft_consensus.cc:3037] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.531191 13665 raft_consensus.cc:348] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.532554 13665 raft_consensus.cc:374] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.532814 13665 raft_consensus.cc:729] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.531461 13671 raft_consensus.cc:686] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:31.533890 13671 consensus_queue.cc:237] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.531208 13617 raft_consensus.cc:2454] T a6234f05657b4c0fbb21339b1a63d571 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:31.537459 13665 consensus_queue.cc:260] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.539557 13665 ts_tablet_manager.cc:1398] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240503 08:04:31.540445 13665 tablet_bootstrap.cc:492] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.545295 13689 raft_consensus.cc:686] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:31.546056 13680 raft_consensus.cc:482] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.546285 13689 consensus_queue.cc:237] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.546569 13680 raft_consensus.cc:504] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.557102 13661 ts_tablet_manager.cc:1398] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.159s user 0.055s sys 0.062s
I20240503 08:04:31.548352 13710 raft_consensus.cc:686] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:31.559300 13661 tablet_bootstrap.cc:492] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.553436 13677 raft_consensus.cc:504] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.560488 13757 raft_consensus.cc:482] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.560981 13757 raft_consensus.cc:504] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.562192 13677 leader_election.cc:288] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.563066 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1b40442f084e05b58edb1d96de3fa3" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:31.563354 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1b40442f084e05b58edb1d96de3fa3" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:31.564023 13546 raft_consensus.cc:3037] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:31.565217 13585 leader_election.cc:341] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.566255 13661 tablet_bootstrap.cc:654] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.567690 13665 tablet_bootstrap.cc:654] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.556113 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ac96e14f7469dba60c7f68d25e882" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.573009 13752 consensus_queue.cc:1035] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
W20240503 08:04:31.558110 13434 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 58974.6us
I20240503 08:04:31.576023 13680 leader_election.cc:288] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.600816 13660 tablet_bootstrap.cc:492] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.575330 13710 consensus_queue.cc:237] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.602303 13660 ts_tablet_manager.cc:1367] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.138s user 0.010s sys 0.006s
I20240503 08:04:31.578804 13471 raft_consensus.cc:2454] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:31.620282 13585 leader_election.cc:302] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.607251 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f41e425e564506958c79d20cc661e1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.613965 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ac96e14f7469dba60c7f68d25e882" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.613000 13546 raft_consensus.cc:2454] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:31.634650 13545 raft_consensus.cc:2454] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:31.618398 13757 leader_election.cc:288] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.640486 13689 raft_consensus.cc:482] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.661152 13469 raft_consensus.cc:2454] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:31.663030 13585 leader_election.cc:302] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.660939 13689 raft_consensus.cc:504] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.641458 13757 raft_consensus.cc:2781] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.682984 13757 raft_consensus.cc:482] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.700093 13369 catalog_manager.cc:5484] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:31.698371 13757 raft_consensus.cc:3037] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.709939 13661 tablet_bootstrap.cc:492] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.710335 13661 ts_tablet_manager.cc:1367] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.152s user 0.023s sys 0.027s
I20240503 08:04:31.712476 13661 raft_consensus.cc:348] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.713182 13661 raft_consensus.cc:374] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.713431 13661 raft_consensus.cc:729] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.714154 13661 consensus_queue.cc:260] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.684554 13751 consensus_queue.cc:1035] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20240503 08:04:31.717384 13665 tablet_bootstrap.cc:492] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.718451 13665 ts_tablet_manager.cc:1367] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.178s user 0.013s sys 0.007s
I20240503 08:04:31.705338 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f41e425e564506958c79d20cc661e1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.720854 13661 ts_tablet_manager.cc:1398] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20240503 08:04:31.722205 13689 leader_election.cc:288] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.722548 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fca1878b3e47838c773a0ea155e22b" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:31.723837 13434 leader_election.cc:341] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.728066 13661 tablet_bootstrap.cc:492] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.732136 13660 raft_consensus.cc:348] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.732743 13660 raft_consensus.cc:374] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.732941 13660 raft_consensus.cc:729] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.733438 13660 consensus_queue.cc:260] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.728327 13545 raft_consensus.cc:2454] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:31.726646 13583 leader_election.cc:302] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.720384 13677 raft_consensus.cc:2781] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.737012 13677 raft_consensus.cc:482] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.737422 13677 raft_consensus.cc:3037] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.737622 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fca1878b3e47838c773a0ea155e22b" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:31.739106 13437 leader_election.cc:341] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.739549 13437 leader_election.cc:302] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:31.747385 13754 raft_consensus.cc:2726] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.753439 13680 raft_consensus.cc:2781] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.753983 13680 raft_consensus.cc:686] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:31.754778 13680 consensus_queue.cc:237] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.755288 13757 raft_consensus.cc:504] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.757637 13757 leader_election.cc:288] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.761616 13660 ts_tablet_manager.cc:1398] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.158s user 0.006s sys 0.000s
I20240503 08:04:31.762553 13660 tablet_bootstrap.cc:492] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.761889 13369 catalog_manager.cc:5484] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:31.766541 13369 catalog_manager.cc:5484] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:31.768361 13665 raft_consensus.cc:348] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.769814 13665 raft_consensus.cc:374] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.770015 13665 raft_consensus.cc:729] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.770994 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ac96e14f7469dba60c7f68d25e882" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:31.770458 13665 consensus_queue.cc:260] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.771745 13544 raft_consensus.cc:3037] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.773758 13757 raft_consensus.cc:482] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.774281 13757 raft_consensus.cc:504] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.775015 13665 ts_tablet_manager.cc:1398] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.056s user 0.000s sys 0.006s
I20240503 08:04:31.776707 13751 consensus_queue.cc:1035] T 07fd7638c4454552a7aeb0aa85bef63b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:31.782299 13757 leader_election.cc:288] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.786345 13762 raft_consensus.cc:482] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.787372 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2dd11388594890a81f290c0bac6840" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.787120 13762 raft_consensus.cc:504] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.791612 13661 tablet_bootstrap.cc:654] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.793969 13762 leader_election.cc:288] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.794091 13677 raft_consensus.cc:504] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.800906 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2dd11388594890a81f290c0bac6840" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.801084 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161465c92bbf4897bc9944c2b9869a9a" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.801796 13546 raft_consensus.cc:2379] T ff2dd11388594890a81f290c0bac6840 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:31.803046 13665 tablet_bootstrap.cc:492] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
W20240503 08:04:31.802999 13583 leader_election.cc:341] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.815531 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ac96e14f7469dba60c7f68d25e882" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:31.819510 13470 raft_consensus.cc:3037] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.819612 13585 leader_election.cc:302] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.821259 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f41e425e564506958c79d20cc661e1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:31.822834 13467 raft_consensus.cc:3037] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.807054 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161465c92bbf4897bc9944c2b9869a9a" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
W20240503 08:04:31.829542 13585 leader_election.cc:341] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.830166 13585 leader_election.cc:302] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.831274 13757 raft_consensus.cc:3037] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.835845 13762 raft_consensus.cc:2726] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.830580 13470 raft_consensus.cc:2454] T d04ac96e14f7469dba60c7f68d25e882 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:31.836866 13677 leader_election.cc:288] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.845546 13544 raft_consensus.cc:2454] T d04ac96e14f7469dba60c7f68d25e882 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:31.847335 13585 leader_election.cc:302] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.841840 13368 catalog_manager.cc:5484] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:31.850433 13677 raft_consensus.cc:2781] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.850955 13677 raft_consensus.cc:686] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:31.851938 13677 consensus_queue.cc:237] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.851094 13665 tablet_bootstrap.cc:654] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.857412 13762 raft_consensus.cc:482] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.857910 13762 raft_consensus.cc:504] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.859359 13757 raft_consensus.cc:2726] T ff2dd11388594890a81f290c0bac6840 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.859455 13762 leader_election.cc:288] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:31.860365 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ca0d7dd59d4881b4c9e6a2a6a8a505" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.874128 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f41e425e564506958c79d20cc661e1" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:31.875027 13544 raft_consensus.cc:3037] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.875178 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ca0d7dd59d4881b4c9e6a2a6a8a505" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:31.878576 13545 raft_consensus.cc:2379] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:31.879683 13583 leader_election.cc:302] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:31.882256 13620 raft_consensus.cc:3037] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.882411 13660 tablet_bootstrap.cc:654] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.885856 13369 catalog_manager.cc:5484] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:31.869385 13467 raft_consensus.cc:2454] T 75f41e425e564506958c79d20cc661e1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:31.888854 13585 leader_election.cc:302] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.889891 13762 raft_consensus.cc:2781] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.890410 13762 raft_consensus.cc:686] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:31.892333 13661 tablet_bootstrap.cc:492] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.892786 13661 ts_tablet_manager.cc:1367] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.165s user 0.022s sys 0.018s
I20240503 08:04:31.896962 13661 raft_consensus.cc:348] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.892038 13762 consensus_queue.cc:237] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.897670 13661 raft_consensus.cc:374] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.897961 13661 raft_consensus.cc:729] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.898562 13661 consensus_queue.cc:260] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.902351 13661 ts_tablet_manager.cc:1398] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20240503 08:04:31.902411 13544 raft_consensus.cc:2454] T 75f41e425e564506958c79d20cc661e1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:31.903705 13661 tablet_bootstrap.cc:492] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:31.908874 13710 raft_consensus.cc:482] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:31.911234 13710 raft_consensus.cc:504] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.913249 13710 leader_election.cc:288] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.915840 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1a896bcf18443fb65ff2a59c8dfe3f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:31.916534 13467 raft_consensus.cc:2454] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:31.917551 13754 consensus_queue.cc:1035] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:31.920090 13511 leader_election.cc:302] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.917819 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1a896bcf18443fb65ff2a59c8dfe3f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:31.924755 13512 leader_election.cc:341] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:31.924077 13710 raft_consensus.cc:2781] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:31.925704 13710 raft_consensus.cc:482] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:31.926048 13710 raft_consensus.cc:3037] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.927846 13660 tablet_bootstrap.cc:492] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:31.928200 13660 ts_tablet_manager.cc:1367] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.166s user 0.016s sys 0.002s
I20240503 08:04:31.930670 13757 raft_consensus.cc:2726] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:31.932490 13671 consensus_queue.cc:1035] T e3ca0d7dd59d4881b4c9e6a2a6a8a505 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:31.933622 13661 tablet_bootstrap.cc:654] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.934908 13660 raft_consensus.cc:348] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.935834 13660 raft_consensus.cc:374] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.936053 13660 raft_consensus.cc:729] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.937158 13665 tablet_bootstrap.cc:492] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:31.937539 13665 ts_tablet_manager.cc:1367] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.135s user 0.012s sys 0.008s
I20240503 08:04:31.936537 13660 consensus_queue.cc:260] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.941396 13665 raft_consensus.cc:348] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.942061 13665 raft_consensus.cc:374] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:31.942301 13665 raft_consensus.cc:729] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:31.947198 13660 ts_tablet_manager.cc:1398] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.019s user 0.004s sys 0.002s
I20240503 08:04:31.945708 13665 consensus_queue.cc:260] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.949642 13665 ts_tablet_manager.cc:1398] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240503 08:04:31.951646 13665 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:31.951467 13710 raft_consensus.cc:504] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.952694 13368 catalog_manager.cc:5484] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:31.954620 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1a896bcf18443fb65ff2a59c8dfe3f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:31.955375 13467 raft_consensus.cc:3037] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.957211 13710 leader_election.cc:288] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:31.956632 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1a896bcf18443fb65ff2a59c8dfe3f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:31.958534 13617 raft_consensus.cc:3037] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:31.948139 13660 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:31.956117 13368 catalog_manager.cc:5484] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:31.971310 13467 raft_consensus.cc:2454] T 8d1a896bcf18443fb65ff2a59c8dfe3f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:31.972543 13511 leader_election.cc:302] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:31.973806 13675 raft_consensus.cc:2781] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:31.974316 13675 raft_consensus.cc:686] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:31.975167 13675 consensus_queue.cc:237] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:31.980257 13665 tablet_bootstrap.cc:654] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.989588 13617 raft_consensus.cc:2454] T 8d1a896bcf18443fb65ff2a59c8dfe3f P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
W20240503 08:04:31.993202 13434 consensus_peers.cc:485] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:31.994587 13660 tablet_bootstrap.cc:654] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:31.995069 13661 tablet_bootstrap.cc:492] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:31.995573 13661 ts_tablet_manager.cc:1367] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.092s user 0.020s sys 0.010s
I20240503 08:04:31.999516 13661 raft_consensus.cc:348] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.000459 13661 raft_consensus.cc:374] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.000784 13661 raft_consensus.cc:729] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.001549 13661 consensus_queue.cc:260] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.007373 13661 ts_tablet_manager.cc:1398] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20240503 08:04:32.008394 13661 tablet_bootstrap.cc:492] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.010174 13369 catalog_manager.cc:5484] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:32.014473 13661 tablet_bootstrap.cc:654] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.024200 13665 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.024678 13665 ts_tablet_manager.cc:1367] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.073s user 0.012s sys 0.005s
I20240503 08:04:32.027179 13665 raft_consensus.cc:348] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.027915 13665 raft_consensus.cc:374] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.028192 13665 raft_consensus.cc:729] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.030887 13660 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.031414 13660 ts_tablet_manager.cc:1367] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.083s user 0.010s sys 0.008s
I20240503 08:04:32.031049 13754 consensus_queue.cc:1035] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:32.033311 13660 raft_consensus.cc:348] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.033859 13660 raft_consensus.cc:374] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.034092 13660 raft_consensus.cc:729] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.035498 13660 consensus_queue.cc:260] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.037400 13660 ts_tablet_manager.cc:1398] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240503 08:04:32.038434 13660 tablet_bootstrap.cc:492] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.040936 13661 tablet_bootstrap.cc:492] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.041479 13661 ts_tablet_manager.cc:1367] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.008s
I20240503 08:04:32.043663 13660 tablet_bootstrap.cc:654] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.044078 13661 raft_consensus.cc:348] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.044850 13661 raft_consensus.cc:374] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.045179 13661 raft_consensus.cc:729] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.045861 13661 consensus_queue.cc:260] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.046880 13665 consensus_queue.cc:260] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.048074 13661 ts_tablet_manager.cc:1398] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:32.049245 13665 ts_tablet_manager.cc:1398] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20240503 08:04:32.050155 13665 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.051256 13753 consensus_queue.cc:1035] T ff2dd11388594890a81f290c0bac6840 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.049237 13661 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.061877 13661 tablet_bootstrap.cc:654] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.063803 13665 tablet_bootstrap.cc:654] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.069922 13753 raft_consensus.cc:482] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.070591 13753 raft_consensus.cc:504] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.074765 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248eecaf8416442794c9558a2d73bdc9" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:32.077320 13437 leader_election.cc:341] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.079239 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248eecaf8416442794c9558a2d73bdc9" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.080237 13545 raft_consensus.cc:2454] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:32.082088 13434 leader_election.cc:302] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:32.083313 13671 raft_consensus.cc:2781] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.083838 13671 raft_consensus.cc:482] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.084306 13671 raft_consensus.cc:3037] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.092576 13753 leader_election.cc:288] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.107100 13675 consensus_queue.cc:1035] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:32.119911 13748 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240503 08:04:32.130343 13661 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.130870 13661 ts_tablet_manager.cc:1367] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.028s
I20240503 08:04:32.133337 13660 tablet_bootstrap.cc:492] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.133288 13661 raft_consensus.cc:348] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.133868 13660 ts_tablet_manager.cc:1367] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.096s user 0.022s sys 0.050s
I20240503 08:04:32.134066 13661 raft_consensus.cc:374] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.134330 13661 raft_consensus.cc:729] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.135035 13661 consensus_queue.cc:260] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.137555 13660 raft_consensus.cc:348] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.138516 13660 raft_consensus.cc:374] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.138841 13660 raft_consensus.cc:729] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.139394 13661 ts_tablet_manager.cc:1398] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:32.140333 13661 tablet_bootstrap.cc:492] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.144927 13665 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.145407 13665 ts_tablet_manager.cc:1367] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.095s user 0.017s sys 0.029s
I20240503 08:04:32.145165 13660 consensus_queue.cc:260] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.147259 13660 ts_tablet_manager.cc:1398] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.013s user 0.005s sys 0.003s
I20240503 08:04:32.148159 13660 tablet_bootstrap.cc:492] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.152946 13751 consensus_queue.cc:1035] T a6234f05657b4c0fbb21339b1a63d571 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:32.153600 13661 tablet_bootstrap.cc:654] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.155771 13748 raft_consensus.cc:482] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.156267 13748 raft_consensus.cc:504] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.157928 13665 raft_consensus.cc:348] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.159039 13665 raft_consensus.cc:374] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.159395 13665 raft_consensus.cc:729] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.159662 13660 tablet_bootstrap.cc:654] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.159340 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1b40442f084e05b58edb1d96de3fa3" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.159934 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b1b40442f084e05b58edb1d96de3fa3" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.160717 13545 raft_consensus.cc:2379] T 4b1b40442f084e05b58edb1d96de3fa3 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
I20240503 08:04:32.160594 13665 consensus_queue.cc:260] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.162128 13748 leader_election.cc:288] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.163376 13434 leader_election.cc:302] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:32.161824 13671 raft_consensus.cc:504] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:32.164609 13434 consensus_peers.cc:485] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:32.165624 13665 ts_tablet_manager.cc:1398] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20240503 08:04:32.178653 13665 tablet_bootstrap.cc:492] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.167753 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248eecaf8416442794c9558a2d73bdc9" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.179906 13544 raft_consensus.cc:3037] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.168491 13753 consensus_queue.cc:1035] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.176455 13660 tablet_bootstrap.cc:492] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.177047 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248eecaf8416442794c9558a2d73bdc9" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:32.184484 13618 raft_consensus.cc:3037] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.189750 13544 raft_consensus.cc:2454] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:32.191398 13660 ts_tablet_manager.cc:1367] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.010s
I20240503 08:04:32.191562 13434 leader_election.cc:302] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.192540 13753 raft_consensus.cc:2781] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.193310 13753 raft_consensus.cc:686] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:32.193939 13660 raft_consensus.cc:348] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.194685 13660 raft_consensus.cc:374] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.194165 13753 consensus_queue.cc:237] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.190891 13618 raft_consensus.cc:2454] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:32.194973 13660 raft_consensus.cc:729] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.196151 13660 consensus_queue.cc:260] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.198490 13660 ts_tablet_manager.cc:1398] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.007s user 0.001s sys 0.003s
I20240503 08:04:32.179147 13671 leader_election.cc:288] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.166335 13748 raft_consensus.cc:3037] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.201352 13661 tablet_bootstrap.cc:492] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.201833 13661 ts_tablet_manager.cc:1367] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.062s user 0.019s sys 0.004s
I20240503 08:04:32.204277 13661 raft_consensus.cc:348] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.204999 13661 raft_consensus.cc:374] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.205271 13661 raft_consensus.cc:729] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.205957 13661 consensus_queue.cc:260] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.207957 13665 tablet_bootstrap.cc:654] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.208207 13661 ts_tablet_manager.cc:1398] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20240503 08:04:32.209069 13661 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.209970 13660 tablet_bootstrap.cc:492] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.212957 13753 raft_consensus.cc:482] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.214089 13753 raft_consensus.cc:504] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.220342 13762 raft_consensus.cc:482] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.220829 13762 raft_consensus.cc:504] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.221585 13748 raft_consensus.cc:2726] T 4b1b40442f084e05b58edb1d96de3fa3 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:32.222986 13369 catalog_manager.cc:5484] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:32.218506 13660 tablet_bootstrap.cc:654] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.226094 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.226652 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.222319 13753 leader_election.cc:288] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:32.228117 13434 leader_election.cc:341] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.228183 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0786fc1af8ae401ebb5190aa117ad461" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.229168 13762 leader_election.cc:288] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.229288 13471 raft_consensus.cc:2454] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:32.232244 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0786fc1af8ae401ebb5190aa117ad461" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:32.233291 13437 leader_election.cc:341] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.233739 13437 leader_election.cc:302] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:32.235417 13661 tablet_bootstrap.cc:654] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.235282 13585 leader_election.cc:302] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
W20240503 08:04:32.235949 13583 leader_election.cc:341] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.236220 13677 raft_consensus.cc:2781] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.238209 13748 raft_consensus.cc:2726] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:32.238413 13677 raft_consensus.cc:482] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.235844 13757 consensus_queue.cc:1035] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.238819 13677 raft_consensus.cc:3037] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.240545 13660 tablet_bootstrap.cc:492] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.248620 13660 ts_tablet_manager.cc:1367] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.013s
I20240503 08:04:32.251021 13660 raft_consensus.cc:348] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.252434 13660 raft_consensus.cc:374] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.252718 13660 raft_consensus.cc:729] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.253432 13660 consensus_queue.cc:260] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.255674 13660 ts_tablet_manager.cc:1398] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240503 08:04:32.256757 13660 tablet_bootstrap.cc:492] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.269878 13660 tablet_bootstrap.cc:654] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.270247 13677 raft_consensus.cc:504] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.274158 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0786fc1af8ae401ebb5190aa117ad461" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.274981 13467 raft_consensus.cc:3037] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.277388 13757 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.278066 13757 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.282146 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.284278 13665 tablet_bootstrap.cc:492] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
W20240503 08:04:32.284425 13583 leader_election.cc:341] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.285235 13751 raft_consensus.cc:482] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.285882 13751 raft_consensus.cc:504] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.286680 13757 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.291625 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0786fc1af8ae401ebb5190aa117ad461" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.284768 13665 ts_tablet_manager.cc:1367] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.106s user 0.012s sys 0.005s
I20240503 08:04:32.297694 13665 raft_consensus.cc:348] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.298683 13665 raft_consensus.cc:374] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.298875 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e609185a44d14a5473954db083b1a" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.298228 13751 leader_election.cc:288] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:32.302357 13583 leader_election.cc:341] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.304618 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e609185a44d14a5473954db083b1a" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.305099 13762 consensus_queue.cc:1035] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:32.307147 13512 leader_election.cc:302] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:32.310323 13751 raft_consensus.cc:2726] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:32.308367 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.311609 13471 raft_consensus.cc:2454] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:32.313442 13585 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:32.314684 13762 raft_consensus.cc:2781] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.315057 13762 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.315663 13762 raft_consensus.cc:3037] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.284889 13677 leader_election.cc:288] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.318739 13467 raft_consensus.cc:2454] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:32.324261 13585 leader_election.cc:302] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:32.299060 13665 raft_consensus.cc:729] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.325636 13765 raft_consensus.cc:2781] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.326113 13765 raft_consensus.cc:686] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:32.326898 13765 consensus_queue.cc:237] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.329432 13757 consensus_queue.cc:1035] T d04ac96e14f7469dba60c7f68d25e882 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.320120 13677 consensus_queue.cc:1035] T 4b1b40442f084e05b58edb1d96de3fa3 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:32.322291 13661 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.331846 13661 ts_tablet_manager.cc:1367] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.123s user 0.035s sys 0.037s
I20240503 08:04:32.334280 13661 raft_consensus.cc:348] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.334971 13661 raft_consensus.cc:374] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.335268 13661 raft_consensus.cc:729] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.335840 13661 consensus_queue.cc:260] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.337858 13661 ts_tablet_manager.cc:1398] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:32.338987 13661 tablet_bootstrap.cc:492] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.340601 13665 consensus_queue.cc:260] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.344719 13661 tablet_bootstrap.cc:654] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.348074 13660 tablet_bootstrap.cc:492] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.348775 13660 ts_tablet_manager.cc:1367] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.092s user 0.045s sys 0.015s
I20240503 08:04:32.352231 13660 raft_consensus.cc:348] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.353461 13660 raft_consensus.cc:374] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.353804 13660 raft_consensus.cc:729] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.354629 13660 consensus_queue.cc:260] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.356127 13661 tablet_bootstrap.cc:492] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.356626 13661 ts_tablet_manager.cc:1367] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20240503 08:04:32.356937 13660 ts_tablet_manager.cc:1398] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20240503 08:04:32.358031 13660 tablet_bootstrap.cc:492] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.358659 13665 ts_tablet_manager.cc:1398] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.065s user 0.008s sys 0.000s
I20240503 08:04:32.359558 13661 raft_consensus.cc:348] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.360258 13661 raft_consensus.cc:374] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.360518 13661 raft_consensus.cc:729] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.361110 13661 consensus_queue.cc:260] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.363492 13661 ts_tablet_manager.cc:1398] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20240503 08:04:32.364573 13660 tablet_bootstrap.cc:654] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.364709 13661 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.371762 13661 tablet_bootstrap.cc:654] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.379087 13544 raft_consensus.cc:3037] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.360772 13762 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.390908 13665 tablet_bootstrap.cc:492] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.391764 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.392627 13467 raft_consensus.cc:3037] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.406673 13467 raft_consensus.cc:2454] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:32.408236 13585 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.409512 13762 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.410223 13762 raft_consensus.cc:2781] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.410792 13762 raft_consensus.cc:686] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:32.417507 13762 consensus_queue.cc:237] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.425359 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.419922 13757 consensus_queue.cc:1035] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20240503 08:04:32.438468 13367 catalog_manager.cc:5484] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
W20240503 08:04:32.449294 13583 leader_election.cc:341] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.458789 13665 tablet_bootstrap.cc:654] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.459837 13710 consensus_queue.cc:1035] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.472514 13660 tablet_bootstrap.cc:492] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.472638 13661 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.473167 13660 ts_tablet_manager.cc:1367] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.115s user 0.043s sys 0.045s
I20240503 08:04:32.473191 13661 ts_tablet_manager.cc:1367] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.109s user 0.046s sys 0.052s
I20240503 08:04:32.475960 13661 raft_consensus.cc:348] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.476660 13661 raft_consensus.cc:374] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.476938 13661 raft_consensus.cc:729] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.476210 13660 raft_consensus.cc:348] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.477072 13677 consensus_queue.cc:1035] T 75f41e425e564506958c79d20cc661e1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.477473 13660 raft_consensus.cc:374] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.477949 13660 raft_consensus.cc:729] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.477587 13661 consensus_queue.cc:260] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.478714 13660 consensus_queue.cc:260] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.480572 13661 ts_tablet_manager.cc:1398] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240503 08:04:32.481462 13660 ts_tablet_manager.cc:1398] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240503 08:04:32.481541 13661 tablet_bootstrap.cc:492] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.482419 13660 tablet_bootstrap.cc:492] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
W20240503 08:04:32.504869 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:32.523561 13661 tablet_bootstrap.cc:654] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.527805 13660 tablet_bootstrap.cc:654] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.523820 13367 catalog_manager.cc:5484] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:32.553438 13665 tablet_bootstrap.cc:492] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.553901 13665 ts_tablet_manager.cc:1367] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.163s user 0.013s sys 0.010s
I20240503 08:04:32.553956 13748 raft_consensus.cc:482] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.554616 13748 raft_consensus.cc:504] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.558725 13675 consensus_queue.cc:1035] T 8d1a896bcf18443fb65ff2a59c8dfe3f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.559377 13761 raft_consensus.cc:482] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.559226 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153d55380cbd4262965848a39a01e17d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.559907 13761 raft_consensus.cc:504] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:32.561113 13434 leader_election.cc:341] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.562662 13660 tablet_bootstrap.cc:492] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.563310 13660 ts_tablet_manager.cc:1367] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.081s user 0.013s sys 0.015s
I20240503 08:04:32.575922 13675 raft_consensus.cc:482] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.576683 13675 raft_consensus.cc:504] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.579829 13661 tablet_bootstrap.cc:492] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.580330 13661 ts_tablet_manager.cc:1367] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.099s user 0.023s sys 0.019s
I20240503 08:04:32.581315 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161465c92bbf4897bc9944c2b9869a9a" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.581939 13619 raft_consensus.cc:2454] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:32.582816 13661 raft_consensus.cc:348] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.583559 13661 raft_consensus.cc:374] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.583849 13661 raft_consensus.cc:729] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.583428 13512 leader_election.cc:302] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:32.585704 13751 raft_consensus.cc:2781] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.586117 13751 raft_consensus.cc:482] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.585824 13665 raft_consensus.cc:348] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.586483 13751 raft_consensus.cc:3037] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.586668 13665 raft_consensus.cc:374] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.588351 13661 consensus_queue.cc:260] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.589844 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161465c92bbf4897bc9944c2b9869a9a" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.579881 13675 leader_election.cc:288] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.590512 13468 raft_consensus.cc:2454] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:32.591961 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805134c1c2c84e2eab2b59a184b7abd4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.592689 13661 ts_tablet_manager.cc:1398] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20240503 08:04:32.592602 13471 raft_consensus.cc:2454] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:32.593622 13661 tablet_bootstrap.cc:492] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.594367 13585 leader_election.cc:302] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.595147 13677 raft_consensus.cc:2781] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.595504 13677 raft_consensus.cc:482] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.595877 13677 raft_consensus.cc:3037] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.593101 13761 leader_election.cc:288] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.597755 13748 leader_election.cc:288] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.598277 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153d55380cbd4262965848a39a01e17d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.598915 13619 raft_consensus.cc:2454] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:32.600050 13437 leader_election.cc:302] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:32.602077 13748 raft_consensus.cc:2781] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.602306 13661 tablet_bootstrap.cc:654] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.602574 13748 raft_consensus.cc:482] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.603263 13748 raft_consensus.cc:3037] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.587002 13665 raft_consensus.cc:729] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.605252 13677 raft_consensus.cc:504] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.606176 13665 consensus_queue.cc:260] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.608654 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805134c1c2c84e2eab2b59a184b7abd4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.609262 13471 raft_consensus.cc:3037] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.609365 13753 raft_consensus.cc:482] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.609869 13753 raft_consensus.cc:504] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.610463 13665 ts_tablet_manager.cc:1398] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.056s user 0.006s sys 0.002s
I20240503 08:04:32.611382 13665 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.611714 13753 leader_election.cc:288] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.614101 13751 raft_consensus.cc:504] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.615948 13751 leader_election.cc:288] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.616699 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161465c92bbf4897bc9944c2b9869a9a" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.617022 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f58e3d02a7d451c90bc97b49b1f83dd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.617297 13468 raft_consensus.cc:3037] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.617537 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161465c92bbf4897bc9944c2b9869a9a" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:32.618155 13618 raft_consensus.cc:3037] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.618453 13753 consensus_queue.cc:1035] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 08:04:32.620457 13437 leader_election.cc:341] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.625180 13468 raft_consensus.cc:2454] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:32.625023 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805134c1c2c84e2eab2b59a184b7abd4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.626464 13511 leader_election.cc:302] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.626757 13677 leader_election.cc:288] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.627607 13751 raft_consensus.cc:2781] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.628620 13751 raft_consensus.cc:686] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:32.633325 13751 consensus_queue.cc:237] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.629535 13471 raft_consensus.cc:2454] T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:32.638082 13585 leader_election.cc:302] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.639928 13677 raft_consensus.cc:2781] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.640398 13677 raft_consensus.cc:686] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:32.641237 13677 consensus_queue.cc:237] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:32.643218 13583 leader_election.cc:341] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.644904 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f58e3d02a7d451c90bc97b49b1f83dd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:32.647558 13434 leader_election.cc:341] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.648370 13434 leader_election.cc:302] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:32.649716 13753 raft_consensus.cc:2726] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:32.652549 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805134c1c2c84e2eab2b59a184b7abd4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.654954 13665 tablet_bootstrap.cc:654] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.657454 13660 raft_consensus.cc:348] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.658553 13660 raft_consensus.cc:374] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.658952 13660 raft_consensus.cc:729] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
W20240503 08:04:32.664286 13583 leader_election.cc:341] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.665233 13618 raft_consensus.cc:2454] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:32.667054 13368 catalog_manager.cc:5484] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:32.669685 13749 consensus_queue.cc:1035] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.678007 13665 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.678874 13665 ts_tablet_manager.cc:1367] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.068s user 0.011s sys 0.001s
I20240503 08:04:32.682312 13660 consensus_queue.cc:260] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.683279 13761 raft_consensus.cc:482] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.683933 13761 raft_consensus.cc:504] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.684667 13660 ts_tablet_manager.cc:1398] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.121s user 0.007s sys 0.000s
I20240503 08:04:32.685694 13660 tablet_bootstrap.cc:492] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.687263 13761 leader_election.cc:288] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.699661 13661 tablet_bootstrap.cc:492] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.700615 13661 ts_tablet_manager.cc:1367] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.107s user 0.016s sys 0.036s
I20240503 08:04:32.700182 13368 catalog_manager.cc:5484] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:32.701650 13660 tablet_bootstrap.cc:654] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.703048 13661 raft_consensus.cc:348] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.703867 13661 raft_consensus.cc:374] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.704142 13661 raft_consensus.cc:729] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.689237 13748 raft_consensus.cc:504] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.689622 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fca1878b3e47838c773a0ea155e22b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.705524 13467 raft_consensus.cc:2454] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:32.682519 13665 raft_consensus.cc:348] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.706301 13665 raft_consensus.cc:374] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.706598 13665 raft_consensus.cc:729] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.706637 13748 leader_election.cc:288] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.694234 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fca1878b3e47838c773a0ea155e22b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.707314 13665 consensus_queue.cc:260] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.708176 13585 leader_election.cc:302] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.708544 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153d55380cbd4262965848a39a01e17d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:32.709148 13618 raft_consensus.cc:3037] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.709225 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153d55380cbd4262965848a39a01e17d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.709550 13665 ts_tablet_manager.cc:1398] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20240503 08:04:32.704779 13661 consensus_queue.cc:260] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.710337 13665 tablet_bootstrap.cc:492] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
W20240503 08:04:32.712843 13434 leader_election.cc:341] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:32.713162 13583 leader_election.cc:341] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.714037 13761 raft_consensus.cc:2781] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.714427 13761 raft_consensus.cc:482] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.714759 13761 raft_consensus.cc:3037] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.719214 13618 raft_consensus.cc:2454] T 153d55380cbd4262965848a39a01e17d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:32.721174 13437 leader_election.cc:302] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:32.721900 13761 raft_consensus.cc:504] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.722370 13748 raft_consensus.cc:2781] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.723188 13748 raft_consensus.cc:686] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:32.724614 13761 leader_election.cc:288] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.724551 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fca1878b3e47838c773a0ea155e22b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.725253 13467 raft_consensus.cc:3037] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.726929 13660 tablet_bootstrap.cc:492] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.727409 13748 consensus_queue.cc:237] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.725178 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09fca1878b3e47838c773a0ea155e22b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.712854 13661 ts_tablet_manager.cc:1398] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
W20240503 08:04:32.729475 13583 leader_election.cc:341] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.729702 13661 tablet_bootstrap.cc:492] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.732321 13467 raft_consensus.cc:2454] T 09fca1878b3e47838c773a0ea155e22b P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:32.734068 13585 leader_election.cc:302] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:32.735321 13761 raft_consensus.cc:2781] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.735693 13661 tablet_bootstrap.cc:654] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.736672 13761 raft_consensus.cc:686] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:32.728055 13660 ts_tablet_manager.cc:1367] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.001s
I20240503 08:04:32.737563 13761 consensus_queue.cc:237] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.740430 13660 raft_consensus.cc:348] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.741240 13660 raft_consensus.cc:374] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.741595 13660 raft_consensus.cc:729] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.727306 13665 tablet_bootstrap.cc:654] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.742372 13660 consensus_queue.cc:260] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.747251 13752 raft_consensus.cc:482] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.747787 13752 raft_consensus.cc:504] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.749425 13660 ts_tablet_manager.cc:1398] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20240503 08:04:32.750661 13660 tablet_bootstrap.cc:492] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.755573 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.756516 13467 raft_consensus.cc:2454] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:32.756665 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:32.758338 13512 leader_election.cc:341] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.759068 13752 leader_election.cc:288] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.763784 13665 tablet_bootstrap.cc:492] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.764389 13665 ts_tablet_manager.cc:1367] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.006s
I20240503 08:04:32.765734 13660 tablet_bootstrap.cc:654] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.766564 13511 leader_election.cc:302] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:32.767962 13752 raft_consensus.cc:2781] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.768381 13752 raft_consensus.cc:482] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.768792 13752 raft_consensus.cc:3037] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.783576 13369 catalog_manager.cc:5484] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:32.784513 13752 raft_consensus.cc:504] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.791329 13665 raft_consensus.cc:348] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.792053 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.793733 13467 raft_consensus.cc:3037] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.795415 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
W20240503 08:04:32.797441 13512 leader_election.cc:341] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.801651 13467 raft_consensus.cc:2454] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:32.793187 13752 leader_election.cc:288] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.804792 13661 tablet_bootstrap.cc:492] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.805291 13511 leader_election.cc:302] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:32.806095 13752 raft_consensus.cc:2781] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.807185 13752 raft_consensus.cc:686] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:32.808105 13752 consensus_queue.cc:237] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.812963 13661 ts_tablet_manager.cc:1367] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.083s user 0.012s sys 0.009s
I20240503 08:04:32.815090 13660 tablet_bootstrap.cc:492] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.815554 13660 ts_tablet_manager.cc:1367] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.065s user 0.023s sys 0.034s
I20240503 08:04:32.793654 13665 raft_consensus.cc:374] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.817687 13661 raft_consensus.cc:348] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.818416 13661 raft_consensus.cc:374] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.818706 13661 raft_consensus.cc:729] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.816826 13665 raft_consensus.cc:729] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.818805 13367 catalog_manager.cc:5484] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:32.819388 13661 consensus_queue.cc:260] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.819669 13665 consensus_queue.cc:260] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.821544 13661 ts_tablet_manager.cc:1398] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240503 08:04:32.822391 13661 tablet_bootstrap.cc:492] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.822649 13665 ts_tablet_manager.cc:1398] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.058s user 0.001s sys 0.004s
I20240503 08:04:32.823487 13665 tablet_bootstrap.cc:492] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.824115 13369 catalog_manager.cc:5484] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:32.821213 13660 raft_consensus.cc:348] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.826735 13660 raft_consensus.cc:374] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.827013 13660 raft_consensus.cc:729] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.827742 13660 consensus_queue.cc:260] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.828547 13661 tablet_bootstrap.cc:654] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.830132 13660 ts_tablet_manager.cc:1398] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.014s user 0.001s sys 0.004s
I20240503 08:04:32.831174 13660 tablet_bootstrap.cc:492] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.832125 13665 tablet_bootstrap.cc:654] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.837694 13660 tablet_bootstrap.cc:654] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.843310 13665 tablet_bootstrap.cc:492] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.843750 13665 ts_tablet_manager.cc:1367] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.008s
I20240503 08:04:32.846035 13665 raft_consensus.cc:348] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.846706 13665 raft_consensus.cc:374] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.846967 13665 raft_consensus.cc:729] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.847568 13665 consensus_queue.cc:260] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.849627 13665 ts_tablet_manager.cc:1398] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:32.850500 13665 tablet_bootstrap.cc:492] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.850651 13660 tablet_bootstrap.cc:492] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.850845 13661 tablet_bootstrap.cc:492] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.851150 13660 ts_tablet_manager.cc:1367] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.011s
I20240503 08:04:32.851271 13661 ts_tablet_manager.cc:1367] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.014s
I20240503 08:04:32.854575 13660 raft_consensus.cc:348] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.855336 13660 raft_consensus.cc:374] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.855587 13660 raft_consensus.cc:729] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.855885 13665 tablet_bootstrap.cc:654] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.886811 13660 consensus_queue.cc:260] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.889206 13660 ts_tablet_manager.cc:1398] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.038s user 0.011s sys 0.014s
I20240503 08:04:32.890056 13660 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.896498 13660 tablet_bootstrap.cc:654] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.886474 13661 raft_consensus.cc:348] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.900491 13661 raft_consensus.cc:374] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.900875 13661 raft_consensus.cc:729] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.901796 13661 consensus_queue.cc:260] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.905059 13661 ts_tablet_manager.cc:1398] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.054s user 0.005s sys 0.002s
I20240503 08:04:32.906037 13754 raft_consensus.cc:482] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.906579 13754 raft_consensus.cc:504] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.908288 13660 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:32.908764 13660 ts_tablet_manager.cc:1367] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.002s
I20240503 08:04:32.909278 13754 leader_election.cc:288] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.910990 13660 raft_consensus.cc:348] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.911728 13660 raft_consensus.cc:374] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.911979 13660 raft_consensus.cc:729] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.912246 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f190cd50f6c2434ebf6d8e0d490eafe7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:32.912917 13761 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.912743 13660 consensus_queue.cc:260] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.913949 13661 tablet_bootstrap.cc:492] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.914006 13677 raft_consensus.cc:482] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.914592 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f190cd50f6c2434ebf6d8e0d490eafe7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:32.914559 13677 raft_consensus.cc:504] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.916651 13677 leader_election.cc:288] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.917841 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def4168780ca4a61aab9436355acca14" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:32.918187 13434 leader_election.cc:341] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.918500 13544 raft_consensus.cc:2454] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:32.919005 13619 raft_consensus.cc:2454] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:32.920092 13583 leader_election.cc:302] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:32.920789 13661 tablet_bootstrap.cc:654] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.921290 13437 leader_election.cc:302] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:32.914870 13660 ts_tablet_manager.cc:1398] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20240503 08:04:32.922955 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def4168780ca4a61aab9436355acca14" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.923651 13471 raft_consensus.cc:2454] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:32.924046 13749 raft_consensus.cc:2781] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.923106 13757 raft_consensus.cc:2781] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:32.924363 13660 tablet_bootstrap.cc:492] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:32.924383 13749 raft_consensus.cc:482] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.924942 13749 raft_consensus.cc:3037] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.924506 13757 raft_consensus.cc:482] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:32.926020 13757 raft_consensus.cc:3037] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.931613 13660 tablet_bootstrap.cc:654] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:32.939817 13665 tablet_bootstrap.cc:492] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:32.940606 13665 ts_tablet_manager.cc:1367] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.090s user 0.025s sys 0.015s
I20240503 08:04:32.944427 13543 raft_consensus.cc:3037] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.953536 13757 raft_consensus.cc:504] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.953881 13749 raft_consensus.cc:504] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.956141 13757 leader_election.cc:288] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.956888 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def4168780ca4a61aab9436355acca14" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:32.957118 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def4168780ca4a61aab9436355acca14" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.957495 13471 raft_consensus.cc:3037] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.957726 13544 raft_consensus.cc:3037] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.964937 13757 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:32.964579 13471 raft_consensus.cc:2454] T def4168780ca4a61aab9436355acca14 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:32.967288 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f190cd50f6c2434ebf6d8e0d490eafe7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:32.967758 13585 leader_election.cc:302] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:32.968152 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f190cd50f6c2434ebf6d8e0d490eafe7" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:32.968721 13619 raft_consensus.cc:3037] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:32.968742 13677 raft_consensus.cc:2781] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:32.969411 13677 raft_consensus.cc:686] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
W20240503 08:04:32.969564 13434 leader_election.cc:341] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:32.964500 13544 raft_consensus.cc:2454] T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:32.974006 13661 tablet_bootstrap.cc:492] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:32.974419 13661 ts_tablet_manager.cc:1367] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.061s user 0.022s sys 0.010s
I20240503 08:04:32.976250 13661 raft_consensus.cc:348] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.979651 13661 raft_consensus.cc:374] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.980760 13665 raft_consensus.cc:348] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.979990 13661 raft_consensus.cc:729] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.981483 13665 raft_consensus.cc:374] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:32.981865 13665 raft_consensus.cc:729] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:32.982441 13665 consensus_queue.cc:260] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.971606 13749 leader_election.cc:288] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:32.976735 13677 consensus_queue.cc:237] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.987165 13675 raft_consensus.cc:482] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.988468 13675 raft_consensus.cc:504] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.987285 13661 consensus_queue.cc:260] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.989269 13665 ts_tablet_manager.cc:1398] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.048s user 0.001s sys 0.007s
I20240503 08:04:32.995370 13661 ts_tablet_manager.cc:1398] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.021s user 0.005s sys 0.003s
I20240503 08:04:32.996016 13749 raft_consensus.cc:482] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.996338 13661 tablet_bootstrap.cc:492] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:32.996884 13762 raft_consensus.cc:482] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:32.996822 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6f8cbdf9a94be9a490fc84cf1c3d7e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:32.997435 13762 raft_consensus.cc:504] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:32.995409 13665 tablet_bootstrap.cc:492] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:32.998982 13675 leader_election.cc:288] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.000921 13762 leader_election.cc:288] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:32.996469 13749 raft_consensus.cc:504] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.002463 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6f8cbdf9a94be9a490fc84cf1c3d7e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.003271 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.003413 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63fcf8624f04059b2dd161f9ae967ea" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:33.005721 13467 raft_consensus.cc:2379] T f63fcf8624f04059b2dd161f9ae967ea P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:33.006500 13512 leader_election.cc:302] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:33.011090 13661 tablet_bootstrap.cc:654] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.012110 13675 raft_consensus.cc:2726] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:33.014209 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51cb053f76ea42bd8ed2a6c66895593d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.014956 13619 raft_consensus.cc:2454] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
W20240503 08:04:33.015501 13434 leader_election.cc:341] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.020153 13665 tablet_bootstrap.cc:654] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.023310 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51cb053f76ea42bd8ed2a6c66895593d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.024139 13619 raft_consensus.cc:2454] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:33.024827 13585 leader_election.cc:302] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:33.029417 13437 leader_election.cc:302] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:33.032696 13677 raft_consensus.cc:3037] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.033450 13437 leader_election.cc:302] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:33.032025 13544 raft_consensus.cc:3037] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.034119 13754 raft_consensus.cc:2781] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:33.049823 13749 leader_election.cc:288] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.051000 13748 raft_consensus.cc:2781] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:33.051649 13748 raft_consensus.cc:482] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.050927 13367 catalog_manager.cc:5484] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:33.052138 13748 raft_consensus.cc:3037] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.056042 13754 raft_consensus.cc:686] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:33.057358 13754 consensus_queue.cc:237] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.063892 13660 tablet_bootstrap.cc:492] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:33.068010 13660 ts_tablet_manager.cc:1367] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.144s user 0.014s sys 0.002s
I20240503 08:04:33.068773 13677 raft_consensus.cc:2726] T f63fcf8624f04059b2dd161f9ae967ea P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:33.081455 13677 raft_consensus.cc:482] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.081233 13748 raft_consensus.cc:504] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.081967 13677 raft_consensus.cc:504] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.084439 13677 leader_election.cc:288] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:33.085629 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51cb053f76ea42bd8ed2a6c66895593d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:33.086102 13660 raft_consensus.cc:348] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.086978 13660 raft_consensus.cc:374] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.088305 13660 raft_consensus.cc:729] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
W20240503 08:04:33.088287 13434 leader_election.cc:341] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.090600 13660 consensus_queue.cc:260] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.096626 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e729d022cc745f6b21dc4320432c817" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.097004 13660 ts_tablet_manager.cc:1398] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.023s user 0.004s sys 0.003s
W20240503 08:04:33.098309 13583 leader_election.cc:341] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.099771 13754 raft_consensus.cc:482] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.100327 13754 raft_consensus.cc:504] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.102365 13754 leader_election.cc:288] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.105831 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:33.107262 13434 leader_election.cc:341] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.106896 13677 consensus_queue.cc:1035] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:33.113679 13660 tablet_bootstrap.cc:492] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:33.112669 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.113123 13367 catalog_manager.cc:5484] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:33.148810 13661 tablet_bootstrap.cc:492] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:33.112483 13665 tablet_bootstrap.cc:492] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:33.153193 13661 ts_tablet_manager.cc:1367] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.157s user 0.023s sys 0.070s
I20240503 08:04:33.153554 13665 ts_tablet_manager.cc:1367] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.158s user 0.015s sys 0.033s
I20240503 08:04:33.143724 13660 tablet_bootstrap.cc:654] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.157317 13675 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.155347 13748 leader_election.cc:288] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.144435 13757 consensus_queue.cc:1035] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20240503 08:04:33.142187 13751 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20240503 08:04:33.159801 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e729d022cc745f6b21dc4320432c817" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:33.160867 13675 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:33.167498 13437 leader_election.cc:341] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.169862 13675 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.170714 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:33.171440 13471 raft_consensus.cc:2454] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
W20240503 08:04:33.171789 13583 consensus_peers.cc:485] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:33.173125 13661 raft_consensus.cc:348] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.172650 13511 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:33.173815 13661 raft_consensus.cc:374] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.174396 13661 raft_consensus.cc:729] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.174814 13661 consensus_queue.cc:260] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.168864 13437 leader_election.cc:302] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:33.164515 13467 raft_consensus.cc:2454] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:33.176105 13544 raft_consensus.cc:3037] T ae6f8cbdf9a94be9a490fc84cf1c3d7e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.176141 13751 raft_consensus.cc:2781] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240503 08:04:33.185087 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51cb053f76ea42bd8ed2a6c66895593d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:33.185693 13618 raft_consensus.cc:3037] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.186549 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.190119 13748 raft_consensus.cc:2726] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:33.182643 13751 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.195647 13661 ts_tablet_manager.cc:1398] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.042s user 0.003s sys 0.003s
I20240503 08:04:33.196950 13661 tablet_bootstrap.cc:492] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:33.196883 13751 raft_consensus.cc:3037] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:33.202219 13585 leader_election.cc:302] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:33.203035 13677 raft_consensus.cc:2781] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:33.203485 13677 raft_consensus.cc:482] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.203801 13677 raft_consensus.cc:3037] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.202857 13618 raft_consensus.cc:2454] T 51cb053f76ea42bd8ed2a6c66895593d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
W20240503 08:04:33.209874 13434 consensus_peers.cc:485] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:33.216667 13437 leader_election.cc:302] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:33.217923 13749 raft_consensus.cc:2781] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:33.218405 13749 raft_consensus.cc:686] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:33.219290 13749 consensus_queue.cc:237] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.207300 13843 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
W20240503 08:04:33.251312 13583 consensus_peers.cc:485] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:33.249699 13665 raft_consensus.cc:348] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.253851 13665 raft_consensus.cc:374] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.254120 13665 raft_consensus.cc:729] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.255466 13661 tablet_bootstrap.cc:654] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.257160 13660 tablet_bootstrap.cc:492] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:33.254648 13665 consensus_queue.cc:260] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.265296 13677 raft_consensus.cc:504] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.274806 13660 ts_tablet_manager.cc:1367] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.163s user 0.017s sys 0.004s
I20240503 08:04:33.281322 13660 raft_consensus.cc:348] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.282004 13660 raft_consensus.cc:374] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.282263 13660 raft_consensus.cc:729] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.282835 13660 consensus_queue.cc:260] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.283468 13751 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.284906 13543 raft_consensus.cc:1226] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Rejecting Update request from peer cf70d9fd04894ea3812767cdb12156c5 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20240503 08:04:33.286512 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:33.288714 13757 consensus_queue.cc:1046] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:33.289602 13757 raft_consensus.cc:3032] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 08:04:33.289929 13757 raft_consensus.cc:729] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:33.290717 13757 consensus_queue.cc:260] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.292150 13660 ts_tablet_manager.cc:1398] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.016s user 0.003s sys 0.002s
I20240503 08:04:33.293395 13660 tablet_bootstrap.cc:492] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:33.297106 13665 ts_tablet_manager.cc:1398] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.143s user 0.004s sys 0.007s
I20240503 08:04:33.291666 13762 consensus_queue.cc:1035] T 805134c1c2c84e2eab2b59a184b7abd4 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20240503 08:04:33.304745 13665 tablet_bootstrap.cc:492] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:33.296554 13757 raft_consensus.cc:3037] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:33.310101 13749 consensus_queue.cc:1035] T 153d55380cbd4262965848a39a01e17d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:33.310539 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:33.308765 13751 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 2 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.311805 13677 leader_election.cc:288] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:33.312028 13470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e729d022cc745f6b21dc4320432c817" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:33.314460 13367 catalog_manager.cc:5484] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:33.317046 13470 raft_consensus.cc:3037] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.360790 13661 tablet_bootstrap.cc:492] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:33.366513 13661 ts_tablet_manager.cc:1367] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.170s user 0.017s sys 0.031s
I20240503 08:04:33.388060 13751 raft_consensus.cc:482] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.407891 13661 raft_consensus.cc:348] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.409612 13661 raft_consensus.cc:374] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.409984 13661 raft_consensus.cc:729] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.410892 13661 consensus_queue.cc:260] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.416663 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e729d022cc745f6b21dc4320432c817" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
W20240503 08:04:33.421404 13583 leader_election.cc:341] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.421841 13842 consensus_queue.cc:1035] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:33.409945 13677 consensus_queue.cc:1035] T 09fca1878b3e47838c773a0ea155e22b P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20240503 08:04:33.405794 13751 raft_consensus.cc:504] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.433764 13661 ts_tablet_manager.cc:1398] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.067s user 0.004s sys 0.003s
I20240503 08:04:33.434736 13661 tablet_bootstrap.cc:492] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:33.440346 13751 leader_election.cc:288] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.441110 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f58e3d02a7d451c90bc97b49b1f83dd" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:33.441789 13468 raft_consensus.cc:2454] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:33.441988 13470 raft_consensus.cc:2454] T 0e729d022cc745f6b21dc4320432c817 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:33.445142 13511 leader_election.cc:302] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:33.449488 13689 raft_consensus.cc:482] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.450019 13689 raft_consensus.cc:504] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.450413 13843 raft_consensus.cc:2781] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:33.450939 13843 raft_consensus.cc:482] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.451421 13843 raft_consensus.cc:3037] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.452534 13689 leader_election.cc:288] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.455520 13620 raft_consensus.cc:2396] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20240503 08:04:33.456317 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f58e3d02a7d451c90bc97b49b1f83dd" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.467496 13615 raft_consensus.cc:2454] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:33.457134 13585 leader_election.cc:302] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:33.470415 13757 raft_consensus.cc:2781] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:33.471743 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7e36891053477ba558573130590a0f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.471932 13757 raft_consensus.cc:686] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:33.472427 13543 raft_consensus.cc:2454] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:33.473760 13512 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:33.474076 13752 consensus_queue.cc:1035] T f63fcf8624f04059b2dd161f9ae967ea P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:33.474931 13842 raft_consensus.cc:2726] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240503 08:04:33.473158 13757 consensus_queue.cc:237] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.476529 13660 tablet_bootstrap.cc:654] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.480571 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7e36891053477ba558573130590a0f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:33.483230 13437 leader_election.cc:341] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.490859 13843 raft_consensus.cc:504] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.491806 13434 leader_election.cc:302] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:33.494709 13749 raft_consensus.cc:2781] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:33.495573 13749 raft_consensus.cc:482] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.495954 13749 raft_consensus.cc:3037] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.496037 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f58e3d02a7d451c90bc97b49b1f83dd" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:33.605784 13843 leader_election.cc:288] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.615075 13665 tablet_bootstrap.cc:654] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.515398 13753 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.093s
I20240503 08:04:33.629909 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f58e3d02a7d451c90bc97b49b1f83dd" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:33.630897 13615 raft_consensus.cc:3037] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.621512 13466 raft_consensus.cc:3037] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.640226 13369 catalog_manager.cc:5484] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:33.672771 13661 tablet_bootstrap.cc:654] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.680954 13749 raft_consensus.cc:504] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.683066 13749 leader_election.cc:288] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:33.683776 13853 log.cc:924] Time spent T 805134c1c2c84e2eab2b59a184b7abd4 P f2462cc996fb441a847521d550b300a4: Append to log took a long time: real 0.118s user 0.000s sys 0.000s
I20240503 08:04:33.697428 13660 tablet_bootstrap.cc:492] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:33.698031 13660 ts_tablet_manager.cc:1367] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.405s user 0.069s sys 0.052s
I20240503 08:04:33.700525 13660 raft_consensus.cc:348] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:33.701146 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:33.701205 13660 raft_consensus.cc:374] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.704871 13660 raft_consensus.cc:729] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.705619 13660 consensus_queue.cc:260] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.715281 13660 ts_tablet_manager.cc:1398] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20240503 08:04:33.716068 13660 tablet_bootstrap.cc:492] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:33.720431 13677 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.720953 13677 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.722839 13677 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:33.724817 13615 raft_consensus.cc:2454] T 0f58e3d02a7d451c90bc97b49b1f83dd P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:33.743245 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7e36891053477ba558573130590a0f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:33.745610 13677 consensus_queue.cc:1035] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:33.749783 13861 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.750527 13861 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.753212 13861 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.771718 13466 raft_consensus.cc:2454] T 0f58e3d02a7d451c90bc97b49b1f83dd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:33.772974 13511 leader_election.cc:302] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:33.774209 13751 raft_consensus.cc:2781] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
W20240503 08:04:33.772979 13437 leader_election.cc:341] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.774827 13751 raft_consensus.cc:686] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:33.776762 13751 consensus_queue.cc:237] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.771121 13660 tablet_bootstrap.cc:654] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:33.786737 13661 tablet_bootstrap.cc:492] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:33.787453 13661 ts_tablet_manager.cc:1367] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.353s user 0.009s sys 0.014s
I20240503 08:04:33.822126 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:33.822943 13466 raft_consensus.cc:2454] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:33.826038 13757 consensus_queue.cc:1035] T def4168780ca4a61aab9436355acca14 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:33.853132 13665 tablet_bootstrap.cc:492] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:33.853736 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.854427 13615 raft_consensus.cc:2454] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:33.859879 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7e36891053477ba558573130590a0f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:33.860512 13544 raft_consensus.cc:3037] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.860687 13661 raft_consensus.cc:348] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.861583 13661 raft_consensus.cc:374] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.861939 13661 raft_consensus.cc:729] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.862668 13661 consensus_queue.cc:260] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.853757 13665 ts_tablet_manager.cc:1367] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.549s user 0.020s sys 0.065s
I20240503 08:04:33.878957 13660 tablet_bootstrap.cc:492] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:33.879657 13660 ts_tablet_manager.cc:1367] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.164s user 0.028s sys 0.022s
I20240503 08:04:33.884636 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.879009 13544 raft_consensus.cc:2454] T ad7e36891053477ba558573130590a0f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:33.886893 13437 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:33.887534 13585 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:33.889428 13861 raft_consensus.cc:2781] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:33.889751 13757 raft_consensus.cc:2781] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:33.889824 13861 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.890316 13757 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:33.891363 13757 raft_consensus.cc:3037] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.893606 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.898473 13661 ts_tablet_manager.cc:1398] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.111s user 0.004s sys 0.007s
I20240503 08:04:33.899776 13661 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:33.894627 13861 raft_consensus.cc:3037] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:33.908694 13660 raft_consensus.cc:348] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.909543 13660 raft_consensus.cc:374] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.909857 13660 raft_consensus.cc:729] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.910578 13660 consensus_queue.cc:260] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.919986 13660 ts_tablet_manager.cc:1398] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.040s user 0.007s sys 0.001s
I20240503 08:04:33.921355 13660 tablet_bootstrap.cc:492] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:33.931908 13754 raft_consensus.cc:482] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.932458 13754 raft_consensus.cc:504] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.934422 13754 leader_election.cc:288] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.944089 13665 raft_consensus.cc:348] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.944660 13665 raft_consensus.cc:374] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:33.944869 13665 raft_consensus.cc:729] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:33.948228 13665 consensus_queue.cc:260] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.949090 13751 raft_consensus.cc:482] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.949549 13751 raft_consensus.cc:504] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:33.951598 13434 consensus_peers.cc:485] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:33.951694 13757 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:33.952615 13583 leader_election.cc:341] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:33.953217 13434 leader_election.cc:341] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.953837 13757 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:33.954167 13434 leader_election.cc:302] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:33.955221 13751 leader_election.cc:288] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:33.957703 13748 raft_consensus.cc:2781] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:33.958186 13748 raft_consensus.cc:686] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:33.958315 13665 ts_tablet_manager.cc:1398] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.091s user 0.001s sys 0.005s
I20240503 08:04:33.959226 13665 tablet_bootstrap.cc:492] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:33.959823 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153d55380cbd4262965848a39a01e17d" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:33.965068 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.959005 13748 consensus_queue.cc:237] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.967917 13861 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.968920 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153d55380cbd4262965848a39a01e17d" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:33.972210 13860 consensus_queue.cc:1035] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240503 08:04:33.973831 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:33.974925 13368 catalog_manager.cc:5484] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
W20240503 08:04:33.975909 13583 consensus_peers.cc:485] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:33.977385 13762 raft_consensus.cc:482] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:33.977840 13762 raft_consensus.cc:504] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:33.979035 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:33.980018 13861 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:33.981354 13583 leader_election.cc:341] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.981727 13762 leader_election.cc:288] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
W20240503 08:04:33.982290 13434 consensus_peers.cc:485] T 51cb053f76ea42bd8ed2a6c66895593d P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:33.982833 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04443187a214cc6ad2bd52022d10822" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:33.985918 13434 leader_election.cc:341] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.991904 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
W20240503 08:04:33.991390 13583 leader_election.cc:341] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.992763 13471 raft_consensus.cc:2379] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:33.992825 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:33.988037 13512 leader_election.cc:302] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:33.994236 13751 raft_consensus.cc:2726] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240503 08:04:33.994962 13434 leader_election.cc:341] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:33.997717 13661 tablet_bootstrap.cc:654] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:34.006075 13437 leader_election.cc:341] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:34.009579 13869 log.cc:924] Time spent T def4168780ca4a61aab9436355acca14 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20240503 08:04:34.006654 13437 leader_election.cc:302] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:34.010825 13861 raft_consensus.cc:2726] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.010746 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:34.011736 13618 raft_consensus.cc:2379] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
I20240503 08:04:34.028044 13437 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:34.033337 13367 catalog_manager.cc:5484] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:34.033360 13660 tablet_bootstrap.cc:654] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.036608 13859 raft_consensus.cc:2726] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.071643 13665 tablet_bootstrap.cc:654] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.079265 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04443187a214cc6ad2bd52022d10822" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.080312 13665 tablet_bootstrap.cc:492] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.080767 13665 ts_tablet_manager.cc:1367] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.122s user 0.024s sys 0.021s
I20240503 08:04:34.083320 13665 raft_consensus.cc:348] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.084096 13665 raft_consensus.cc:374] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.084371 13665 raft_consensus.cc:729] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.085589 13585 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.086429 13757 raft_consensus.cc:2726] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.086390 13665 consensus_queue.cc:260] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:34.087098 13585 leader_election.cc:341] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.087582 13585 leader_election.cc:302] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.088641 13850 raft_consensus.cc:2726] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.090395 13761 consensus_queue.cc:1035] T 0e729d022cc745f6b21dc4320432c817 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:34.092352 13665 ts_tablet_manager.cc:1398] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.011s user 0.001s sys 0.005s
I20240503 08:04:34.093219 13665 tablet_bootstrap.cc:492] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.098568 13665 tablet_bootstrap.cc:654] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.104341 13665 tablet_bootstrap.cc:492] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.104876 13665 ts_tablet_manager.cc:1367] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20240503 08:04:34.107537 13665 raft_consensus.cc:348] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.108242 13665 raft_consensus.cc:374] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.108554 13665 raft_consensus.cc:729] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.109251 13665 consensus_queue.cc:260] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.111212 13665 ts_tablet_manager.cc:1398] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:34.111989 13660 tablet_bootstrap.cc:492] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.112097 13665 tablet_bootstrap.cc:492] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.112790 13660 ts_tablet_manager.cc:1367] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.192s user 0.007s sys 0.014s
I20240503 08:04:34.115795 13660 raft_consensus.cc:348] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.116698 13660 raft_consensus.cc:374] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.117003 13660 raft_consensus.cc:729] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.118432 13665 tablet_bootstrap.cc:654] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.117689 13660 consensus_queue.cc:260] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.131289 13660 ts_tablet_manager.cc:1398] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20240503 08:04:34.132200 13660 tablet_bootstrap.cc:492] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:34.136582 13661 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:34.137061 13661 ts_tablet_manager.cc:1367] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.238s user 0.014s sys 0.002s
I20240503 08:04:34.138295 13660 tablet_bootstrap.cc:654] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.139286 13661 raft_consensus.cc:348] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.139994 13661 raft_consensus.cc:374] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.140262 13661 raft_consensus.cc:729] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.140868 13661 consensus_queue.cc:260] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.142951 13661 ts_tablet_manager.cc:1398] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:34.143883 13665 tablet_bootstrap.cc:492] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.144436 13665 ts_tablet_manager.cc:1367] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.011s
I20240503 08:04:34.143900 13661 tablet_bootstrap.cc:492] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:34.147430 13665 raft_consensus.cc:348] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.148213 13665 raft_consensus.cc:374] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.148489 13665 raft_consensus.cc:729] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.149060 13665 consensus_queue.cc:260] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.151310 13665 ts_tablet_manager.cc:1398] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20240503 08:04:34.152315 13665 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.153306 13661 tablet_bootstrap.cc:654] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.163645 13665 tablet_bootstrap.cc:654] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.169018 13751 consensus_queue.cc:1035] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:34.175062 13660 tablet_bootstrap.cc:492] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.175087 13665 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.179414 13660 ts_tablet_manager.cc:1367] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.004s
I20240503 08:04:34.179713 13665 ts_tablet_manager.cc:1367] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.001s
I20240503 08:04:34.182080 13660 raft_consensus.cc:348] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.189594 13660 raft_consensus.cc:374] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.189970 13660 raft_consensus.cc:729] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.190570 13661 tablet_bootstrap.cc:492] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:34.190680 13660 consensus_queue.cc:260] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.191088 13661 ts_tablet_manager.cc:1367] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.047s user 0.015s sys 0.004s
I20240503 08:04:34.182698 13665 raft_consensus.cc:348] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.186336 13843 consensus_queue.cc:1035] T 0f58e3d02a7d451c90bc97b49b1f83dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:34.192199 13665 raft_consensus.cc:374] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.192531 13665 raft_consensus.cc:729] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.193341 13660 ts_tablet_manager.cc:1398] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.013s user 0.004s sys 0.001s
I20240503 08:04:34.193229 13665 consensus_queue.cc:260] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.194923 13660 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:34.195302 13665 ts_tablet_manager.cc:1398] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20240503 08:04:34.196168 13665 tablet_bootstrap.cc:492] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.201823 13665 tablet_bootstrap.cc:654] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.201982 13660 tablet_bootstrap.cc:654] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.204820 13546 raft_consensus.cc:3037] T 153d55380cbd4262965848a39a01e17d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.207201 13661 raft_consensus.cc:348] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.207965 13661 raft_consensus.cc:374] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.208245 13661 raft_consensus.cc:729] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.219862 13843 raft_consensus.cc:482] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.220432 13843 raft_consensus.cc:504] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.222443 13843 leader_election.cc:288] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.224498 13661 consensus_queue.cc:260] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.227448 13665 tablet_bootstrap.cc:492] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.227964 13665 ts_tablet_manager.cc:1367] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.011s
I20240503 08:04:34.228052 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:34.229408 13512 leader_election.cc:341] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.230753 13665 raft_consensus.cc:348] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.231097 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.231540 13665 raft_consensus.cc:374] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.231768 13469 raft_consensus.cc:2454] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:34.231986 13665 raft_consensus.cc:729] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.232900 13661 ts_tablet_manager.cc:1398] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.041s user 0.008s sys 0.000s
I20240503 08:04:34.232863 13665 consensus_queue.cc:260] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.235656 13661 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:34.236264 13665 ts_tablet_manager.cc:1398] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:34.237144 13665 tablet_bootstrap.cc:492] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.237179 13660 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.237778 13660 ts_tablet_manager.cc:1367] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.004s
I20240503 08:04:34.233003 13511 leader_election.cc:302] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:34.241410 13660 raft_consensus.cc:348] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.242174 13660 raft_consensus.cc:374] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.242383 13752 raft_consensus.cc:482] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.242439 13660 raft_consensus.cc:729] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.243086 13752 raft_consensus.cc:504] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.243633 13665 tablet_bootstrap.cc:654] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.243588 13660 consensus_queue.cc:260] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.245208 13752 leader_election.cc:288] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.245705 13752 raft_consensus.cc:2781] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.246024 13752 raft_consensus.cc:482] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.246248 13660 ts_tablet_manager.cc:1398] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:34.247668 13660 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:34.249773 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f190cd50f6c2434ebf6d8e0d490eafe7" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.246336 13752 raft_consensus.cc:3037] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.269752 13677 raft_consensus.cc:482] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.270258 13677 raft_consensus.cc:504] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.273065 13661 tablet_bootstrap.cc:654] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.272260 13677 leader_election.cc:288] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:34.322034 13752 raft_consensus.cc:504] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.326674 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.328100 13752 leader_election.cc:288] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.277931 13544 raft_consensus.cc:3037] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.323390 13762 raft_consensus.cc:482] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.280560 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.329032 13469 raft_consensus.cc:2454] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:34.328871 13762 raft_consensus.cc:504] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.330308 13660 tablet_bootstrap.cc:654] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.330466 13585 leader_election.cc:302] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:34.331218 13665 tablet_bootstrap.cc:492] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.331696 13665 ts_tablet_manager.cc:1367] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.095s user 0.018s sys 0.019s
I20240503 08:04:34.274564 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f190cd50f6c2434ebf6d8e0d490eafe7" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.332628 13762 leader_election.cc:288] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:34.334210 13677 raft_consensus.cc:2781] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.335386 13677 raft_consensus.cc:482] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.335841 13677 raft_consensus.cc:3037] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.334781 13665 raft_consensus.cc:348] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.336496 13665 raft_consensus.cc:374] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.336817 13665 raft_consensus.cc:729] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.338331 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.337896 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.337478 13665 consensus_queue.cc:260] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:34.339790 13583 leader_election.cc:341] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:34.340183 13585 leader_election.cc:341] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.341735 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
W20240503 08:04:34.342972 13583 leader_election.cc:341] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.343320 13469 raft_consensus.cc:3037] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.343477 13583 leader_election.cc:302] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.344563 13762 raft_consensus.cc:2726] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.346068 13665 ts_tablet_manager.cc:1398] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20240503 08:04:34.345830 13512 leader_election.cc:302] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.347086 13665 tablet_bootstrap.cc:492] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.348656 13843 raft_consensus.cc:2726] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.357223 13677 raft_consensus.cc:504] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.357393 13762 raft_consensus.cc:482] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.358573 13762 raft_consensus.cc:504] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.360462 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:34.361083 13466 raft_consensus.cc:3037] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.362514 13762 leader_election.cc:288] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:34.364228 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac21dd494074d0fa7690d8d182585f4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
W20240503 08:04:34.366457 13585 leader_election.cc:341] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.368242 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac21dd494074d0fa7690d8d182585f4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.368933 13545 raft_consensus.cc:2454] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:34.369418 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.371022 13583 leader_election.cc:302] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.371878 13761 raft_consensus.cc:2781] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.372329 13761 raft_consensus.cc:482] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.372684 13761 raft_consensus.cc:3037] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.377735 13677 leader_election.cc:288] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
W20240503 08:04:34.380580 13583 leader_election.cc:341] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.379689 13466 raft_consensus.cc:2454] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:34.383639 13585 leader_election.cc:302] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:34.343870 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:34.385233 13677 raft_consensus.cc:2781] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
W20240503 08:04:34.388109 13512 leader_election.cc:341] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.388150 13677 raft_consensus.cc:686] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:34.389905 13677 consensus_queue.cc:237] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.393812 13469 raft_consensus.cc:2454] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:34.392082 13761 raft_consensus.cc:504] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.400714 13511 leader_election.cc:302] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:34.427779 13761 leader_election.cc:288] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:34.433720 13660 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.434895 13660 ts_tablet_manager.cc:1367] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.188s user 0.013s sys 0.003s
I20240503 08:04:34.435081 13752 raft_consensus.cc:2781] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:34.437335 13660 raft_consensus.cc:348] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.437666 13752 raft_consensus.cc:686] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:34.438023 13660 raft_consensus.cc:374] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.438354 13660 raft_consensus.cc:729] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.439065 13660 consensus_queue.cc:260] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.441028 13660 ts_tablet_manager.cc:1398] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:34.442943 13842 raft_consensus.cc:482] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2462cc996fb441a847521d550b300a4)
I20240503 08:04:34.445027 13660 tablet_bootstrap.cc:492] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:34.444716 13842 raft_consensus.cc:504] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.443428 13546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac21dd494074d0fa7690d8d182585f4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.451848 13546 raft_consensus.cc:3037] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.452538 13368 catalog_manager.cc:5484] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:34.456802 13842 leader_election.cc:288] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.460538 13546 raft_consensus.cc:2454] T cac21dd494074d0fa7690d8d182585f4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:34.464121 13583 leader_election.cc:302] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:34.467525 13752 consensus_queue.cc:237] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.433676 13665 tablet_bootstrap.cc:654] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.520268 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51cb053f76ea42bd8ed2a6c66895593d" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.521663 13661 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:34.522050 13661 ts_tablet_manager.cc:1367] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.287s user 0.019s sys 0.014s
I20240503 08:04:34.525118 13761 raft_consensus.cc:2781] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:34.528626 13665 tablet_bootstrap.cc:492] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.529098 13665 ts_tablet_manager.cc:1367] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.182s user 0.011s sys 0.008s
I20240503 08:04:34.530094 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac21dd494074d0fa7690d8d182585f4" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
W20240503 08:04:34.533421 13585 leader_election.cc:341] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.535151 13859 raft_consensus.cc:482] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.531253 13660 tablet_bootstrap.cc:654] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.537065 13661 raft_consensus.cc:348] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.547441 13859 raft_consensus.cc:504] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.539525 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51cb053f76ea42bd8ed2a6c66895593d" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.549559 13859 leader_election.cc:288] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.541239 13689 consensus_queue.cc:1035] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20240503 08:04:34.550309 13661 raft_consensus.cc:374] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.550199 13511 leader_election.cc:302] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.541366 13761 raft_consensus.cc:686] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:34.535931 13368 catalog_manager.cc:5484] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:34.551699 13761 consensus_queue.cc:237] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.552310 13752 raft_consensus.cc:2726] T 51cb053f76ea42bd8ed2a6c66895593d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240503 08:04:34.541498 13583 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 66564.4us
I20240503 08:04:34.556006 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8497edff1aaf45e48da16ad73a649c6d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:34.556151 13437 consensus_peers.cc:485] T ad7e36891053477ba558573130590a0f P f2462cc996fb441a847521d550b300a4 -> Peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Couldn't send request to peer cf70d9fd04894ea3812767cdb12156c5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:34.557056 13546 raft_consensus.cc:3037] T 0e729d022cc745f6b21dc4320432c817 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.561834 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8497edff1aaf45e48da16ad73a649c6d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.550757 13661 raft_consensus.cc:729] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.562536 13618 raft_consensus.cc:2454] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:34.562963 13661 consensus_queue.cc:260] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.567158 13437 leader_election.cc:302] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:34.569665 13689 raft_consensus.cc:2781] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.570290 13689 raft_consensus.cc:482] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.570842 13689 raft_consensus.cc:3037] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.575387 13661 ts_tablet_manager.cc:1398] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.053s user 0.008s sys 0.000s
W20240503 08:04:34.579661 13434 leader_election.cc:341] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.581629 13689 raft_consensus.cc:504] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.583743 13661 tablet_bootstrap.cc:492] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:34.538533 13665 raft_consensus.cc:348] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.584196 13665 raft_consensus.cc:374] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.584441 13665 raft_consensus.cc:729] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.585067 13665 consensus_queue.cc:260] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.604460 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8497edff1aaf45e48da16ad73a649c6d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.605912 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8497edff1aaf45e48da16ad73a649c6d" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:34.606453 13618 raft_consensus.cc:3037] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.597885 13665 ts_tablet_manager.cc:1398] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.068s user 0.003s sys 0.003s
I20240503 08:04:34.608726 13665 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.610515 13661 tablet_bootstrap.cc:654] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.602941 13689 leader_election.cc:288] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.616883 13368 catalog_manager.cc:5484] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:34.625325 13618 raft_consensus.cc:2454] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:34.626766 13437 leader_election.cc:302] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:34.627256 13665 tablet_bootstrap.cc:654] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:34.629163 13434 leader_election.cc:341] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.630405 13660 tablet_bootstrap.cc:492] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.630872 13660 ts_tablet_manager.cc:1367] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.186s user 0.035s sys 0.030s
I20240503 08:04:34.633512 13754 raft_consensus.cc:2781] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:34.634011 13754 raft_consensus.cc:686] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:34.634536 13660 raft_consensus.cc:348] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.635474 13660 raft_consensus.cc:374] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.635757 13660 raft_consensus.cc:729] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.636368 13660 consensus_queue.cc:260] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.645838 13660 ts_tablet_manager.cc:1398] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20240503 08:04:34.634810 13754 consensus_queue.cc:237] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.659924 13660 tablet_bootstrap.cc:492] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:34.665302 13544 raft_consensus.cc:3037] T 09fca1878b3e47838c773a0ea155e22b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.679226 13761 raft_consensus.cc:482] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.686147 13675 raft_consensus.cc:482] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.686703 13675 raft_consensus.cc:504] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.688771 13675 leader_election.cc:288] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.690817 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805134c1c2c84e2eab2b59a184b7abd4" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.692024 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805134c1c2c84e2eab2b59a184b7abd4" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.693583 13511 leader_election.cc:302] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.695400 13752 raft_consensus.cc:2726] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.696005 13660 tablet_bootstrap.cc:654] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.681763 13859 raft_consensus.cc:482] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.706075 13542 raft_consensus.cc:3037] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.681385 13761 raft_consensus.cc:504] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.711004 13761 leader_election.cc:288] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:34.703451 13859 raft_consensus.cc:504] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.712116 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.713328 13859 leader_election.cc:288] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.713861 13367 catalog_manager.cc:5484] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:34.719365 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04443187a214cc6ad2bd52022d10822" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:34.720950 13434 leader_election.cc:341] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.723505 13665 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.724058 13665 ts_tablet_manager.cc:1367] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.116s user 0.016s sys 0.024s
I20240503 08:04:34.724643 13859 raft_consensus.cc:482] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.725167 13859 raft_consensus.cc:504] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.727030 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbbe543644940cdab442cd183707b91" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.727909 13469 raft_consensus.cc:2379] T 7bbbe543644940cdab442cd183707b91 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:34.743763 13859 leader_election.cc:288] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.744056 13665 raft_consensus.cc:348] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.750671 13665 raft_consensus.cc:374] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.751121 13665 raft_consensus.cc:729] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.753860 13585 leader_election.cc:302] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.755345 13754 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cf70d9fd04894ea3812767cdb12156c5)
I20240503 08:04:34.755858 13754 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.745195 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d88e8a1e1ad45f9a3cdf022a28162fb" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.760711 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04443187a214cc6ad2bd52022d10822" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.761379 13618 raft_consensus.cc:2454] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:34.762038 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.762967 13541 raft_consensus.cc:2379] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 2: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:34.763077 13761 raft_consensus.cc:3037] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.767067 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d88e8a1e1ad45f9a3cdf022a28162fb" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.758280 13665 consensus_queue.cc:260] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.770202 13437 leader_election.cc:302] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
W20240503 08:04:34.771576 13437 leader_election.cc:341] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.772711 13665 ts_tablet_manager.cc:1398] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.048s user 0.009s sys 0.000s
I20240503 08:04:34.771728 13859 raft_consensus.cc:2781] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.774341 13665 tablet_bootstrap.cc:492] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.771981 13543 raft_consensus.cc:2454] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:34.774410 13859 raft_consensus.cc:482] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.774865 13859 raft_consensus.cc:3037] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.777806 13754 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.781456 13434 leader_election.cc:302] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:34.783264 13754 raft_consensus.cc:2781] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.783687 13754 raft_consensus.cc:482] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.785621 13754 raft_consensus.cc:3037] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.785454 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.786154 13618 raft_consensus.cc:2454] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 2.
I20240503 08:04:34.779315 13757 raft_consensus.cc:482] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.787681 13859 raft_consensus.cc:504] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.789220 13761 raft_consensus.cc:2726] T 7bbbe543644940cdab442cd183707b91 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.787087 13437 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:34.787658 13757 raft_consensus.cc:504] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.792745 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04443187a214cc6ad2bd52022d10822" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.793931 13689 raft_consensus.cc:2781] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240503 08:04:34.794909 13689 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cf70d9fd04894ea3812767cdb12156c5)
W20240503 08:04:34.795262 13434 leader_election.cc:341] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.795466 13689 raft_consensus.cc:3037] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Advancing to term 2
W20240503 08:04:34.811525 13583 consensus_peers.cc:485] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:34.813484 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:34.817353 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04443187a214cc6ad2bd52022d10822" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:34.817863 13618 raft_consensus.cc:3037] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.820708 13859 leader_election.cc:288] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:34.820914 13585 leader_election.cc:341] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.822104 13757 leader_election.cc:288] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:34.827283 13543 raft_consensus.cc:3037] T 805134c1c2c84e2eab2b59a184b7abd4 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.830695 13689 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.832099 13618 raft_consensus.cc:2454] T e04443187a214cc6ad2bd52022d10822 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:34.833216 13437 leader_election.cc:302] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:34.833393 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.834136 13541 raft_consensus.cc:2379] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 2: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:34.834293 13754 raft_consensus.cc:504] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.836071 13754 leader_election.cc:288] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.836565 13754 raft_consensus.cc:2781] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:34.836781 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d88e8a1e1ad45f9a3cdf022a28162fb" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.837033 13754 raft_consensus.cc:686] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:34.837436 13541 raft_consensus.cc:3037] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.837935 13665 tablet_bootstrap.cc:654] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.827797 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:34.840531 13583 leader_election.cc:341] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.841122 13583 leader_election.cc:302] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.846570 13660 tablet_bootstrap.cc:492] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.844113 13761 raft_consensus.cc:2726] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:34.837855 13754 consensus_queue.cc:237] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.848331 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d88e8a1e1ad45f9a3cdf022a28162fb" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
W20240503 08:04:34.849920 13437 leader_election.cc:341] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.850107 13661 tablet_bootstrap.cc:492] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:34.850585 13661 ts_tablet_manager.cc:1367] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.267s user 0.012s sys 0.004s
I20240503 08:04:34.852447 13660 ts_tablet_manager.cc:1367] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.194s user 0.020s sys 0.001s
I20240503 08:04:34.855459 13661 raft_consensus.cc:348] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.856220 13661 raft_consensus.cc:374] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.856468 13661 raft_consensus.cc:729] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.857018 13661 consensus_queue.cc:260] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.860287 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:34.862010 13661 ts_tablet_manager.cc:1398] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.011s user 0.004s sys 0.003s
I20240503 08:04:34.862928 13661 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:34.859233 13689 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 2 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.878285 13618 raft_consensus.cc:2454] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 2.
I20240503 08:04:34.879285 13437 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:34.866693 13541 raft_consensus.cc:2454] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:34.881881 13861 raft_consensus.cc:2781] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:34.881769 13434 leader_election.cc:302] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:34.882257 13861 raft_consensus.cc:686] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [term 2 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:34.882524 13859 raft_consensus.cc:2781] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:34.882961 13859 raft_consensus.cc:686] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:34.883103 13861 consensus_queue.cc:237] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.883854 13859 consensus_queue.cc:237] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.896512 13661 tablet_bootstrap.cc:654] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.896970 13665 tablet_bootstrap.cc:492] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.897670 13665 ts_tablet_manager.cc:1367] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.124s user 0.018s sys 0.013s
I20240503 08:04:34.900102 13665 raft_consensus.cc:348] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.900848 13665 raft_consensus.cc:374] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.900194 13367 catalog_manager.cc:5484] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:34.867332 13660 raft_consensus.cc:348] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.901124 13665 raft_consensus.cc:729] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.902215 13665 consensus_queue.cc:260] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.904434 13661 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:34.904825 13661 ts_tablet_manager.cc:1367] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.001s
I20240503 08:04:34.905506 13665 ts_tablet_manager.cc:1398] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20240503 08:04:34.906535 13665 tablet_bootstrap.cc:492] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.907337 13661 raft_consensus.cc:348] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.907981 13661 raft_consensus.cc:374] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.908195 13661 raft_consensus.cc:729] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.908675 13661 consensus_queue.cc:260] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.910089 13661 ts_tablet_manager.cc:1398] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240503 08:04:34.910780 13661 tablet_bootstrap.cc:492] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:34.911320 13660 raft_consensus.cc:374] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.911836 13660 raft_consensus.cc:729] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.912700 13665 tablet_bootstrap.cc:654] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.912882 13660 consensus_queue.cc:260] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.924103 13661 tablet_bootstrap.cc:654] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.924628 13665 tablet_bootstrap.cc:492] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.925169 13665 ts_tablet_manager.cc:1367] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20240503 08:04:34.928745 13660 ts_tablet_manager.cc:1398] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.076s user 0.008s sys 0.000s
I20240503 08:04:34.929533 13660 tablet_bootstrap.cc:492] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:34.933619 13665 raft_consensus.cc:348] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.934255 13665 raft_consensus.cc:374] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.934479 13665 raft_consensus.cc:729] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.934943 13665 consensus_queue.cc:260] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.937333 13665 ts_tablet_manager.cc:1398] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20240503 08:04:34.938251 13665 tablet_bootstrap.cc:492] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.927878 13754 raft_consensus.cc:482] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.939379 13754 raft_consensus.cc:504] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.939906 13660 tablet_bootstrap.cc:654] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:34.942768 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15d8c1499084cfabc5ed6031b6a22ad" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:34.944258 13665 tablet_bootstrap.cc:654] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:34.945513 13434 leader_election.cc:341] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.947881 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15d8c1499084cfabc5ed6031b6a22ad" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.948544 13620 raft_consensus.cc:2454] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:34.949687 13437 leader_election.cc:302] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:34.950402 13754 leader_election.cc:288] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.951151 13754 raft_consensus.cc:2781] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:34.950208 13367 catalog_manager.cc:5484] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 1 to 2, leader changed from cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 2 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:34.951508 13754 raft_consensus.cc:482] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:34.951860 13754 raft_consensus.cc:3037] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.952514 13367 catalog_manager.cc:5484] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:34.958365 13752 raft_consensus.cc:482] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:34.958739 13754 raft_consensus.cc:504] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.959043 13752 raft_consensus.cc:504] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.962146 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15d8c1499084cfabc5ed6031b6a22ad" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:34.966589 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
W20240503 08:04:34.966878 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:34.967783 13469 raft_consensus.cc:2379] T 4fde14f2672c4c0bb9d76f225efd306e P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
W20240503 08:04:34.968456 13434 leader_election.cc:341] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:34.971441 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fde14f2672c4c0bb9d76f225efd306e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:34.973325 13660 tablet_bootstrap.cc:492] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:34.979331 13512 leader_election.cc:302] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:34.981091 13665 tablet_bootstrap.cc:492] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:34.981369 13843 raft_consensus.cc:3037] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.975409 13754 leader_election.cc:288] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.978578 13675 consensus_queue.cc:1035] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20240503 08:04:34.981707 13665 ts_tablet_manager.cc:1367] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.044s user 0.011s sys 0.010s
I20240503 08:04:34.981107 13660 ts_tablet_manager.cc:1367] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.002s
I20240503 08:04:34.986371 13660 raft_consensus.cc:348] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.987164 13660 raft_consensus.cc:374] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.987418 13660 raft_consensus.cc:729] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.988927 13660 consensus_queue.cc:260] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.988202 13665 raft_consensus.cc:348] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.990105 13665 raft_consensus.cc:374] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:34.990411 13665 raft_consensus.cc:729] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:34.989012 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15d8c1499084cfabc5ed6031b6a22ad" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:34.991323 13620 raft_consensus.cc:3037] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:34.991005 13665 consensus_queue.cc:260] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:34.993400 13665 ts_tablet_manager.cc:1398] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20240503 08:04:34.994290 13665 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:34.975069 13752 leader_election.cc:288] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:34.996438 13675 consensus_queue.cc:1035] T 7bbbe543644940cdab442cd183707b91 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:35.002861 13843 raft_consensus.cc:2726] T 4fde14f2672c4c0bb9d76f225efd306e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.003116 13757 consensus_queue.cc:1035] T 4fde14f2672c4c0bb9d76f225efd306e P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.006506 13665 tablet_bootstrap.cc:654] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.017704 13620 raft_consensus.cc:2454] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:35.019022 13437 leader_election.cc:302] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:35.020087 13754 raft_consensus.cc:2781] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:35.020656 13754 raft_consensus.cc:686] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:35.021601 13754 consensus_queue.cc:237] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.050199 13661 tablet_bootstrap.cc:492] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.050670 13661 ts_tablet_manager.cc:1367] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.140s user 0.018s sys 0.023s
I20240503 08:04:35.052932 13661 raft_consensus.cc:348] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.053627 13661 raft_consensus.cc:374] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.053879 13661 raft_consensus.cc:729] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.057369 13689 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:35.068459 13619 raft_consensus.cc:3037] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.070653 13665 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.071278 13665 ts_tablet_manager.cc:1367] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.077s user 0.046s sys 0.020s
I20240503 08:04:35.054451 13661 consensus_queue.cc:260] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.073688 13665 raft_consensus.cc:348] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.074573 13665 raft_consensus.cc:374] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.074921 13665 raft_consensus.cc:729] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.075692 13661 ts_tablet_manager.cc:1398] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.025s user 0.002s sys 0.003s
I20240503 08:04:35.075762 13665 consensus_queue.cc:260] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.078081 13665 ts_tablet_manager.cc:1398] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:35.081200 13661 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.088479 13843 raft_consensus.cc:482] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.089234 13843 raft_consensus.cc:504] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.094056 13843 leader_election.cc:288] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.097075 13665 tablet_bootstrap.cc:492] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.101116 13661 tablet_bootstrap.cc:654] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.103047 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15d8c1499084cfabc5ed6031b6a22ad" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.109472 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15d8c1499084cfabc5ed6031b6a22ad" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:35.110129 13434 consensus_peers.cc:485] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:35.110656 13660 ts_tablet_manager.cc:1398] T ad7e36891053477ba558573130590a0f P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.125s user 0.006s sys 0.000s
I20240503 08:04:35.112095 13859 raft_consensus.cc:482] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.112689 13615 raft_consensus.cc:2379] T e15d8c1499084cfabc5ed6031b6a22ad P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:35.112583 13859 raft_consensus.cc:504] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.114584 13512 leader_election.cc:302] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.116701 13752 raft_consensus.cc:3037] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.118947 13665 tablet_bootstrap.cc:654] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.120139 13859 leader_election.cc:288] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:35.128954 13585 consensus_peers.cc:485] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 -> Peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Couldn't send request to peer f2462cc996fb441a847521d550b300a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:35.163895 13660 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.165304 13843 raft_consensus.cc:482] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.171329 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.171329 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.170895 13843 raft_consensus.cc:504] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.172109 13615 raft_consensus.cc:2454] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
W20240503 08:04:35.178752 13434 leader_election.cc:341] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.180788 13843 leader_election.cc:288] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.181821 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.186997 13752 raft_consensus.cc:2726] T e15d8c1499084cfabc5ed6031b6a22ad P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.196568 13757 consensus_queue.cc:1035] T cac21dd494074d0fa7690d8d182585f4 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.196691 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e4a81f1b4b4ec69620d613e4273c73" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.196823 13368 catalog_manager.cc:5484] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:35.197816 13620 raft_consensus.cc:2396] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20240503 08:04:35.199739 13437 leader_election.cc:302] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:35.201000 13861 raft_consensus.cc:2781] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:35.201371 13861 raft_consensus.cc:482] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:35.201696 13861 raft_consensus.cc:3037] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.203325 13512 leader_election.cc:302] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.205518 13675 raft_consensus.cc:2726] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240503 08:04:35.207566 13665 tablet_bootstrap.cc:492] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.208060 13665 ts_tablet_manager.cc:1367] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.111s user 0.004s sys 0.009s
I20240503 08:04:35.210230 13660 tablet_bootstrap.cc:654] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.212034 13754 raft_consensus.cc:482] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.212529 13754 raft_consensus.cc:504] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.213537 13665 raft_consensus.cc:348] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.214488 13665 raft_consensus.cc:374] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.217723 13665 raft_consensus.cc:729] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.214437 13754 leader_election.cc:288] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.218441 13665 consensus_queue.cc:260] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.220624 13665 ts_tablet_manager.cc:1398] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20240503 08:04:35.220844 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.221483 13665 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.222226 13861 raft_consensus.cc:504] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:35.222507 13434 leader_election.cc:341] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.223875 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.224529 13615 raft_consensus.cc:2454] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:35.227108 13437 leader_election.cc:302] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:35.228232 13754 raft_consensus.cc:2781] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:35.228708 13754 raft_consensus.cc:482] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:35.229167 13754 raft_consensus.cc:3037] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.230976 13861 leader_election.cc:288] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.236313 13661 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.236821 13661 ts_tablet_manager.cc:1367] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.160s user 0.013s sys 0.003s
I20240503 08:04:35.239679 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:35.239601 13661 raft_consensus.cc:348] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.236809 13665 tablet_bootstrap.cc:654] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.240641 13661 raft_consensus.cc:374] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.240988 13661 raft_consensus.cc:729] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
W20240503 08:04:35.241006 13434 leader_election.cc:341] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.241699 13660 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.242172 13660 ts_tablet_manager.cc:1367] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.078s user 0.012s sys 0.002s
I20240503 08:04:35.245172 13660 raft_consensus.cc:348] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.245893 13660 raft_consensus.cc:374] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.246201 13660 raft_consensus.cc:729] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.245371 13661 consensus_queue.cc:260] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.246896 13660 consensus_queue.cc:260] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.249243 13660 ts_tablet_manager.cc:1398] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20240503 08:04:35.249213 13661 ts_tablet_manager.cc:1398] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.012s user 0.004s sys 0.002s
I20240503 08:04:35.250406 13660 tablet_bootstrap.cc:492] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.251541 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:35.252182 13619 raft_consensus.cc:3037] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.259097 13619 raft_consensus.cc:2454] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:35.275353 13437 leader_election.cc:302] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:35.276132 13748 raft_consensus.cc:2781] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:35.265939 13754 raft_consensus.cc:504] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.276587 13748 raft_consensus.cc:686] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:35.277412 13748 consensus_queue.cc:237] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.278054 13754 leader_election.cc:288] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.268656 13842 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.278640 13665 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.279076 13665 ts_tablet_manager.cc:1367] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.058s user 0.028s sys 0.012s
I20240503 08:04:35.274036 13660 tablet_bootstrap.cc:654] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.278760 13842 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.281785 13665 raft_consensus.cc:348] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.282455 13665 raft_consensus.cc:374] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.282828 13665 raft_consensus.cc:729] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.284500 13842 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.285581 13542 raft_consensus.cc:3037] T e04443187a214cc6ad2bd52022d10822 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.287673 13665 consensus_queue.cc:260] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.290043 13665 ts_tablet_manager.cc:1398] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20240503 08:04:35.290905 13665 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.290915 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:35.292225 13619 raft_consensus.cc:3037] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.298719 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.299577 13615 raft_consensus.cc:2379] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
I20240503 08:04:35.300693 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.302121 13471 raft_consensus.cc:2379] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:35.302655 13859 consensus_queue.cc:1035] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.303848 13660 tablet_bootstrap.cc:492] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.304293 13660 ts_tablet_manager.cc:1367] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.054s user 0.004s sys 0.020s
I20240503 08:04:35.305223 13661 tablet_bootstrap.cc:492] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.307587 13660 raft_consensus.cc:348] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.308365 13660 raft_consensus.cc:374] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.308931 13660 raft_consensus.cc:729] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.309875 13660 consensus_queue.cc:260] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.298707 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:35.312116 13660 ts_tablet_manager.cc:1398] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:35.312459 13661 tablet_bootstrap.cc:654] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.313074 13660 tablet_bootstrap.cc:492] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.313239 13619 raft_consensus.cc:2454] T 4061d55705594102a8b057896c5f86d0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:35.306196 13665 tablet_bootstrap.cc:654] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.314460 13437 leader_election.cc:302] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:35.315320 13748 raft_consensus.cc:2781] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:35.315701 13748 raft_consensus.cc:686] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:35.316558 13748 consensus_queue.cc:237] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.318271 13661 tablet_bootstrap.cc:492] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.318832 13661 ts_tablet_manager.cc:1367] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.001s
I20240503 08:04:35.319746 13660 tablet_bootstrap.cc:654] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:35.320698 13434 leader_election.cc:341] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.321342 13661 raft_consensus.cc:348] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.322075 13661 raft_consensus.cc:374] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.322415 13661 raft_consensus.cc:729] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.322476 13512 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.323498 13927 raft_consensus.cc:3037] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.323371 13661 consensus_queue.cc:260] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.325932 13661 ts_tablet_manager.cc:1398] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20240503 08:04:35.326826 13661 tablet_bootstrap.cc:492] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.330606 13927 raft_consensus.cc:2726] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.337931 13754 raft_consensus.cc:482] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.338413 13754 raft_consensus.cc:504] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.340332 13754 leader_election.cc:288] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.341359 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aae93c20681b4e4fa9b521edbb9e65e0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
W20240503 08:04:35.344393 13437 leader_election.cc:341] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.346369 13665 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.346843 13665 ts_tablet_manager.cc:1367] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.056s user 0.009s sys 0.011s
I20240503 08:04:35.348716 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aae93c20681b4e4fa9b521edbb9e65e0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.354619 13543 raft_consensus.cc:2454] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:35.336133 13661 tablet_bootstrap.cc:654] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.356103 13434 leader_election.cc:302] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:35.357347 13660 tablet_bootstrap.cc:492] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.357822 13660 ts_tablet_manager.cc:1367] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.045s user 0.012s sys 0.004s
I20240503 08:04:35.351112 13689 consensus_queue.cc:1035] T e04443187a214cc6ad2bd52022d10822 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.360276 13660 raft_consensus.cc:348] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.360993 13660 raft_consensus.cc:374] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.361263 13660 raft_consensus.cc:729] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.357725 13754 raft_consensus.cc:2781] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:35.361831 13754 raft_consensus.cc:482] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:35.361876 13660 consensus_queue.cc:260] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.362401 13754 raft_consensus.cc:3037] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.361730 13367 catalog_manager.cc:5484] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:35.363525 13367 catalog_manager.cc:5484] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:35.349753 13665 raft_consensus.cc:348] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.365355 13665 raft_consensus.cc:374] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.365792 13665 raft_consensus.cc:729] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.366955 13665 consensus_queue.cc:260] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.369220 13543 raft_consensus.cc:1261] T e8e4a81f1b4b4ec69620d613e4273c73 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 08:04:35.370827 13859 consensus_queue.cc:1035] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.376260 13660 ts_tablet_manager.cc:1398] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.018s user 0.007s sys 0.002s
I20240503 08:04:35.377233 13660 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.377046 13665 ts_tablet_manager.cc:1398] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.030s user 0.008s sys 0.003s
I20240503 08:04:35.379209 13665 tablet_bootstrap.cc:492] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.382480 13660 tablet_bootstrap.cc:654] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.391822 13754 raft_consensus.cc:504] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.393841 13754 leader_election.cc:288] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.395568 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aae93c20681b4e4fa9b521edbb9e65e0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:35.399709 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aae93c20681b4e4fa9b521edbb9e65e0" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:35.400362 13542 raft_consensus.cc:3037] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:35.400493 13437 leader_election.cc:341] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.410179 13665 tablet_bootstrap.cc:654] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.415709 13542 raft_consensus.cc:2454] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:35.414949 13661 tablet_bootstrap.cc:492] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.424823 13927 raft_consensus.cc:482] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.425526 13927 raft_consensus.cc:504] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.429253 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8497edff1aaf45e48da16ad73a649c6d" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.429766 13434 leader_election.cc:302] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:35.431706 13861 raft_consensus.cc:2781] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:35.432399 13861 raft_consensus.cc:686] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:35.433610 13861 consensus_queue.cc:237] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.421272 13661 ts_tablet_manager.cc:1367] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.095s user 0.018s sys 0.005s
I20240503 08:04:35.436445 13850 raft_consensus.cc:482] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.437124 13850 raft_consensus.cc:504] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.438380 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8497edff1aaf45e48da16ad73a649c6d" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.439919 13512 leader_election.cc:302] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.440696 13842 raft_consensus.cc:2726] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.441236 13850 leader_election.cc:288] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:35.441702 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa42fc3b9d894d398b7002af94aa3f40" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.442090 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa42fc3b9d894d398b7002af94aa3f40" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.443413 13619 raft_consensus.cc:1261] T e8e4a81f1b4b4ec69620d613e4273c73 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20240503 08:04:35.443804 13583 leader_election.cc:341] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:35.444836 13585 leader_election.cc:341] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.445276 13585 leader_election.cc:302] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.445355 13859 consensus_queue.cc:1035] T e8e4a81f1b4b4ec69620d613e4273c73 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:35.448067 13762 raft_consensus.cc:2726] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.437158 13927 leader_election.cc:288] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.450758 13927 raft_consensus.cc:482] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.451347 13927 raft_consensus.cc:504] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.455508 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.457772 13618 raft_consensus.cc:3037] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.458724 13927 leader_election.cc:288] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.459723 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c5d2ce341b749f8b2fd0c4ac23b4018" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.460508 13617 raft_consensus.cc:2379] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:35.460716 13762 raft_consensus.cc:482] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.461169 13762 raft_consensus.cc:504] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.461589 13512 leader_election.cc:302] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.463347 13842 raft_consensus.cc:3037] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.463603 13762 leader_election.cc:288] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:35.464015 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:35.465130 13583 leader_election.cc:341] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.465373 13661 raft_consensus.cc:348] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.466037 13661 raft_consensus.cc:374] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.466308 13661 raft_consensus.cc:729] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.472285 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.474716 13661 consensus_queue.cc:260] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:35.475804 13585 leader_election.cc:341] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.476250 13585 leader_election.cc:302] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.476799 13661 ts_tablet_manager.cc:1398] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20240503 08:04:35.476845 13762 raft_consensus.cc:2726] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.478282 13661 tablet_bootstrap.cc:492] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.486775 13861 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:35.487249 13660 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.489388 13660 ts_tablet_manager.cc:1367] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.112s user 0.034s sys 0.035s
I20240503 08:04:35.489832 13665 tablet_bootstrap.cc:492] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.490296 13665 ts_tablet_manager.cc:1367] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.111s user 0.021s sys 0.021s
I20240503 08:04:35.492614 13660 raft_consensus.cc:348] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.493427 13660 raft_consensus.cc:374] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.493337 13665 raft_consensus.cc:348] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.493849 13660 raft_consensus.cc:729] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.494077 13665 raft_consensus.cc:374] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.494390 13665 raft_consensus.cc:729] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.494978 13661 tablet_bootstrap.cc:654] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.494832 13660 consensus_queue.cc:260] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.495093 13665 consensus_queue.cc:260] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.500069 13665 ts_tablet_manager.cc:1398] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20240503 08:04:35.501312 13665 tablet_bootstrap.cc:492] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.504025 13660 ts_tablet_manager.cc:1398] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20240503 08:04:35.509667 13665 tablet_bootstrap.cc:654] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.510187 13842 raft_consensus.cc:2726] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.510695 13660 tablet_bootstrap.cc:492] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.522650 13859 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240503 08:04:35.527352 13660 tablet_bootstrap.cc:654] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.541584 13861 consensus_queue.cc:1035] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.547942 13367 catalog_manager.cc:5484] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:35.569409 13661 tablet_bootstrap.cc:492] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.570142 13661 ts_tablet_manager.cc:1367] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.092s user 0.015s sys 0.023s
I20240503 08:04:35.577469 13665 tablet_bootstrap.cc:492] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.583750 13660 tablet_bootstrap.cc:492] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.584357 13660 ts_tablet_manager.cc:1367] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.077s user 0.019s sys 0.000s
I20240503 08:04:35.586840 13660 raft_consensus.cc:348] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.588554 13661 raft_consensus.cc:348] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.589084 13661 raft_consensus.cc:374] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.589308 13661 raft_consensus.cc:729] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.589761 13661 consensus_queue.cc:260] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.591665 13661 ts_tablet_manager.cc:1398] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.021s user 0.017s sys 0.002s
I20240503 08:04:35.592445 13661 tablet_bootstrap.cc:492] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.594584 13665 ts_tablet_manager.cc:1367] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.093s user 0.013s sys 0.002s
I20240503 08:04:35.587790 13660 raft_consensus.cc:374] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.600900 13660 raft_consensus.cc:729] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.601544 13660 consensus_queue.cc:260] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.597204 13661 tablet_bootstrap.cc:654] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.598423 13748 consensus_queue.cc:1035] T e15d8c1499084cfabc5ed6031b6a22ad P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:35.603736 13660 ts_tablet_manager.cc:1398] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20240503 08:04:35.604696 13660 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.610169 13660 tablet_bootstrap.cc:654] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.612740 13665 raft_consensus.cc:348] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.613503 13665 raft_consensus.cc:374] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.613765 13665 raft_consensus.cc:729] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.614364 13665 consensus_queue.cc:260] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.619982 13859 raft_consensus.cc:482] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.628829 13859 raft_consensus.cc:504] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.631722 13859 leader_election.cc:288] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.632181 13543 raft_consensus.cc:3037] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.652282 13466 raft_consensus.cc:3037] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.664434 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac21dd494074d0fa7690d8d182585f4" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.667883 13661 tablet_bootstrap.cc:492] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.671070 13661 ts_tablet_manager.cc:1367] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.079s user 0.008s sys 0.006s
I20240503 08:04:35.675418 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac21dd494074d0fa7690d8d182585f4" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.677997 13661 raft_consensus.cc:348] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.678745 13661 raft_consensus.cc:374] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.678999 13661 raft_consensus.cc:729] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.680469 13661 consensus_queue.cc:260] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.684736 13661 ts_tablet_manager.cc:1398] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240503 08:04:35.687285 13859 raft_consensus.cc:482] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.688117 13665 ts_tablet_manager.cc:1398] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.088s user 0.008s sys 0.000s
I20240503 08:04:35.687784 13859 raft_consensus.cc:504] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.689006 13665 tablet_bootstrap.cc:492] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.689711 13859 leader_election.cc:288] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.697072 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0b468643254b1dba30573778f90be8" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:35.698755 13434 leader_election.cc:341] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.685777 13661 tablet_bootstrap.cc:492] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.701174 13660 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.701640 13660 ts_tablet_manager.cc:1367] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.097s user 0.030s sys 0.015s
I20240503 08:04:35.703994 13660 raft_consensus.cc:348] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.704752 13660 raft_consensus.cc:374] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.705042 13660 raft_consensus.cc:729] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.705757 13660 consensus_queue.cc:260] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.707547 13437 leader_election.cc:302] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:35.708094 13660 ts_tablet_manager.cc:1398] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20240503 08:04:35.714327 13859 raft_consensus.cc:2726] T cac21dd494074d0fa7690d8d182585f4 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.714731 13660 tablet_bootstrap.cc:492] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.715276 13850 raft_consensus.cc:482] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.715463 13661 tablet_bootstrap.cc:654] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.715728 13850 raft_consensus.cc:504] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.714655 13665 tablet_bootstrap.cc:654] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.719323 13850 leader_election.cc:288] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:35.755167 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0b468643254b1dba30573778f90be8" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.759958 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "008315f470c24029bd9271aad3c67a09" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.761327 13660 tablet_bootstrap.cc:654] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.770612 13850 raft_consensus.cc:482] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.779866 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "008315f470c24029bd9271aad3c67a09" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.779781 13850 raft_consensus.cc:504] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:35.781885 13583 leader_election.cc:341] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.783774 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aae93c20681b4e4fa9b521edbb9e65e0" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:35.771801 13437 leader_election.cc:341] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.785804 13437 leader_election.cc:302] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:35.781898 13850 leader_election.cc:288] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:35.791008 13689 raft_consensus.cc:2726] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.800751 13545 raft_consensus.cc:2379] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:35.788373 13859 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20240503 08:04:35.813575 13958 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.810361 13689 raft_consensus.cc:482] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240503 08:04:35.846165 13585 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.1:34427, user_credentials={real_user=slave}} blocked reactor thread for 66850.2us
I20240503 08:04:35.828181 13689 raft_consensus.cc:504] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.840189 13958 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.841259 13748 consensus_queue.cc:1035] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20240503 08:04:35.869781 13958 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:35.870227 13689 leader_election.cc:288] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.874872 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.875610 13541 raft_consensus.cc:2454] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:35.875902 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13da943b8b4caa8da56b29cd48f93f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.876725 13583 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:35.876780 13617 raft_consensus.cc:2454] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
W20240503 08:04:35.868266 13585 leader_election.cc:341] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.883611 13585 leader_election.cc:302] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.883625 13660 tablet_bootstrap.cc:492] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:35.884969 13660 ts_tablet_manager.cc:1367] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.170s user 0.017s sys 0.007s
I20240503 08:04:35.887064 13661 tablet_bootstrap.cc:492] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:35.887537 13661 ts_tablet_manager.cc:1367] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.202s user 0.028s sys 0.020s
I20240503 08:04:35.877722 13761 raft_consensus.cc:2781] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240503 08:04:35.887864 13761 raft_consensus.cc:482] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:35.888245 13761 raft_consensus.cc:3037] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:35.890761 13660 raft_consensus.cc:348] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.891520 13660 raft_consensus.cc:374] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.891582 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13da943b8b4caa8da56b29cd48f93f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.885644 13959 raft_consensus.cc:2726] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.893621 13661 raft_consensus.cc:348] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.894323 13661 raft_consensus.cc:374] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.891824 13660 raft_consensus.cc:729] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
W20240503 08:04:35.894456 13434 leader_election.cc:341] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:35.894624 13661 raft_consensus.cc:729] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.895390 13661 consensus_queue.cc:260] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.895939 13437 leader_election.cc:302] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:35.896919 13660 consensus_queue.cc:260] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.897310 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.897914 13468 raft_consensus.cc:2454] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:35.898725 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aae93c20681b4e4fa9b521edbb9e65e0" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.899241 13660 ts_tablet_manager.cc:1398] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.014s user 0.005s sys 0.002s
I20240503 08:04:35.901648 13660 tablet_bootstrap.cc:492] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:35.902745 13665 tablet_bootstrap.cc:492] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:35.903210 13665 ts_tablet_manager.cc:1367] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.214s user 0.013s sys 0.014s
I20240503 08:04:35.905906 13748 raft_consensus.cc:2781] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:35.906302 13748 raft_consensus.cc:482] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:35.906628 13748 raft_consensus.cc:3037] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.915290 13665 raft_consensus.cc:348] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.916057 13665 raft_consensus.cc:374] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:35.916337 13665 raft_consensus.cc:729] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:35.917769 13661 ts_tablet_manager.cc:1398] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20240503 08:04:35.918692 13661 tablet_bootstrap.cc:492] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:35.919802 13665 consensus_queue.cc:260] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:35.921638 13434 consensus_peers.cc:485] T 4061d55705594102a8b057896c5f86d0 P f2462cc996fb441a847521d550b300a4 -> Peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Couldn't send request to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:35.909221 13761 raft_consensus.cc:504] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.923455 13761 leader_election.cc:288] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 2 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:35.925218 13585 leader_election.cc:302] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:35.929576 13850 raft_consensus.cc:3037] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.929594 13754 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:35.936939 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:35.937588 13544 raft_consensus.cc:3037] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:35.939221 13665 ts_tablet_manager.cc:1398] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20240503 08:04:35.941265 13665 tablet_bootstrap.cc:492] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:35.949671 13544 raft_consensus.cc:2454] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 2.
I20240503 08:04:35.949900 13748 raft_consensus.cc:504] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.952103 13748 leader_election.cc:288] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.953918 13752 raft_consensus.cc:482] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.954399 13752 raft_consensus.cc:504] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.954649 13583 leader_election.cc:302] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:35.955647 13762 raft_consensus.cc:2781] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 08:04:35.956336 13762 raft_consensus.cc:686] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [term 2 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:35.956970 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa42fc3b9d894d398b7002af94aa3f40" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:35.957377 13850 raft_consensus.cc:2726] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:35.957645 13466 raft_consensus.cc:2454] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:35.957173 13762 consensus_queue.cc:237] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.958760 13511 leader_election.cc:302] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:35.959460 13660 tablet_bootstrap.cc:654] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.959616 13927 raft_consensus.cc:2781] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:35.959973 13927 raft_consensus.cc:482] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:35.960505 13927 raft_consensus.cc:3037] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.961795 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13da943b8b4caa8da56b29cd48f93f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:35.962663 13543 raft_consensus.cc:3037] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.969766 13661 tablet_bootstrap.cc:654] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.970928 13665 tablet_bootstrap.cc:654] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:35.969630 13752 leader_election.cc:288] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:35.975348 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa42fc3b9d894d398b7002af94aa3f40" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:35.979305 13620 raft_consensus.cc:2454] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:35.986907 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13da943b8b4caa8da56b29cd48f93f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:35.987551 13620 raft_consensus.cc:3037] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:35.993784 13543 raft_consensus.cc:2454] T 5e13da943b8b4caa8da56b29cd48f93f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:35.997550 13675 raft_consensus.cc:482] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:35.998193 13675 raft_consensus.cc:504] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:35.982731 13927 raft_consensus.cc:504] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.000592 13675 leader_election.cc:288] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.002647 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969c8edb68343339904e50c63bbdace" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.002986 13434 leader_election.cc:302] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.006184 13861 raft_consensus.cc:2781] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.006781 13861 raft_consensus.cc:686] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:36.008165 13689 raft_consensus.cc:482] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.008792 13689 raft_consensus.cc:504] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.010962 13861 consensus_queue.cc:237] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.018570 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa42fc3b9d894d398b7002af94aa3f40" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.018985 13689 leader_election.cc:288] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.019354 13617 raft_consensus.cc:3037] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.021145 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969c8edb68343339904e50c63bbdace" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.023232 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa42fc3b9d894d398b7002af94aa3f40" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.024696 13466 raft_consensus.cc:3037] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.026413 13617 raft_consensus.cc:2454] T aa42fc3b9d894d398b7002af94aa3f40 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:36.005189 13927 leader_election.cc:288] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:36.027709 13512 leader_election.cc:341] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:36.031064 13511 leader_election.cc:341] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:36.032178 13512 leader_election.cc:302] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:36.033805 13927 raft_consensus.cc:2726] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:36.037799 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "878ffcf68a1b4726a683f6590d94704c" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:35.994987 13620 raft_consensus.cc:2454] T 5e13da943b8b4caa8da56b29cd48f93f P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:36.044891 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e833906e87754702bba4e5dbde972f86" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.045773 13467 raft_consensus.cc:3037] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Advancing to term 2
I20240503 08:04:36.049901 13748 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 08:04:36.053823 13434 leader_election.cc:341] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:36.061483 13512 leader_election.cc:302] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:36.063552 13927 raft_consensus.cc:2781] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.064730 13927 raft_consensus.cc:686] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:36.065915 13927 consensus_queue.cc:237] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:36.069310 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:36.066356 13467 raft_consensus.cc:2454] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 2.
I20240503 08:04:36.107661 13665 tablet_bootstrap.cc:492] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.108204 13665 ts_tablet_manager.cc:1367] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.167s user 0.023s sys 0.031s
I20240503 08:04:36.105594 13366 catalog_manager.cc:5484] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:36.111212 13665 raft_consensus.cc:348] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.111943 13665 raft_consensus.cc:374] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.112394 13665 raft_consensus.cc:729] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.113436 13665 consensus_queue.cc:260] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.116940 13661 tablet_bootstrap.cc:492] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:36.117429 13661 ts_tablet_manager.cc:1367] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.199s user 0.027s sys 0.031s
I20240503 08:04:36.117487 13665 ts_tablet_manager.cc:1398] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20240503 08:04:36.117888 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "878ffcf68a1b4726a683f6590d94704c" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.118561 13620 raft_consensus.cc:2454] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.120674 13466 raft_consensus.cc:2454] T aa42fc3b9d894d398b7002af94aa3f40 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:36.120782 13661 raft_consensus.cc:348] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.121491 13661 raft_consensus.cc:374] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.121769 13661 raft_consensus.cc:729] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.124711 13661 consensus_queue.cc:260] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.126075 13665 tablet_bootstrap.cc:492] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.133780 13660 tablet_bootstrap.cc:492] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:36.134289 13660 ts_tablet_manager.cc:1367] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.233s user 0.017s sys 0.047s
I20240503 08:04:36.138301 13665 tablet_bootstrap.cc:654] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.143519 13660 raft_consensus.cc:348] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.144450 13660 raft_consensus.cc:374] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.144780 13660 raft_consensus.cc:729] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.146349 13660 consensus_queue.cc:260] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.145310 13665 tablet_bootstrap.cc:492] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.147572 13665 ts_tablet_manager.cc:1367] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.000s
I20240503 08:04:36.148559 13660 ts_tablet_manager.cc:1398] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20240503 08:04:36.138497 13661 ts_tablet_manager.cc:1398] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.021s user 0.004s sys 0.003s
I20240503 08:04:36.149657 13661 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:36.149963 13660 tablet_bootstrap.cc:492] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:36.154170 13437 leader_election.cc:302] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:36.154963 13861 raft_consensus.cc:2781] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.155478 13861 raft_consensus.cc:482] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.156080 13861 raft_consensus.cc:3037] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.156518 13665 raft_consensus.cc:348] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.157310 13665 raft_consensus.cc:374] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.157639 13665 raft_consensus.cc:729] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.158334 13665 consensus_queue.cc:260] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.160471 13665 ts_tablet_manager.cc:1398] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240503 08:04:36.161078 13660 tablet_bootstrap.cc:654] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.162182 13665 tablet_bootstrap.cc:492] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.161154 13661 tablet_bootstrap.cc:654] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.167690 13368 catalog_manager.cc:5484] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:36.173839 13748 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:36.181260 13369 catalog_manager.cc:5484] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 2, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 2 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:36.182303 13665 tablet_bootstrap.cc:654] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.193974 13861 raft_consensus.cc:504] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.196506 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "878ffcf68a1b4726a683f6590d94704c" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
W20240503 08:04:36.198148 13434 leader_election.cc:341] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:36.207029 13665 tablet_bootstrap.cc:492] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.207823 13665 ts_tablet_manager.cc:1367] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.046s user 0.023s sys 0.007s
I20240503 08:04:36.210781 13665 raft_consensus.cc:348] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.211827 13665 raft_consensus.cc:374] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.212222 13665 raft_consensus.cc:729] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.213196 13665 consensus_queue.cc:260] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.216428 13665 ts_tablet_manager.cc:1398] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:36.217829 13665 tablet_bootstrap.cc:492] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.224922 13665 tablet_bootstrap.cc:654] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.227408 13861 leader_election.cc:288] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.234444 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "878ffcf68a1b4726a683f6590d94704c" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.235395 13618 raft_consensus.cc:3037] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.237655 13665 tablet_bootstrap.cc:492] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.238353 13665 ts_tablet_manager.cc:1367] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.009s
I20240503 08:04:36.241614 13665 raft_consensus.cc:348] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.242686 13665 raft_consensus.cc:374] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.243080 13665 raft_consensus.cc:729] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.243995 13665 consensus_queue.cc:260] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.246999 13665 ts_tablet_manager.cc:1398] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20240503 08:04:36.248270 13665 tablet_bootstrap.cc:492] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.255540 13665 tablet_bootstrap.cc:654] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.262234 13660 tablet_bootstrap.cc:492] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:36.262862 13660 ts_tablet_manager.cc:1367] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.113s user 0.020s sys 0.035s
I20240503 08:04:36.248190 13618 raft_consensus.cc:2454] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:36.262317 13665 tablet_bootstrap.cc:492] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.267277 13665 ts_tablet_manager.cc:1367] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.000s
I20240503 08:04:36.265556 13660 raft_consensus.cc:348] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.270366 13660 raft_consensus.cc:374] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.270968 13660 raft_consensus.cc:729] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.271600 13661 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:36.269788 13665 raft_consensus.cc:348] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.271998 13665 raft_consensus.cc:374] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.272073 13661 ts_tablet_manager.cc:1367] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.123s user 0.024s sys 0.028s
I20240503 08:04:36.272343 13665 raft_consensus.cc:729] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.271736 13660 consensus_queue.cc:260] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.275457 13675 raft_consensus.cc:482] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.274226 13665 consensus_queue.cc:260] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.276013 13661 raft_consensus.cc:348] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.277206 13661 raft_consensus.cc:374] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.277812 13661 raft_consensus.cc:729] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.278604 13665 ts_tablet_manager.cc:1398] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20240503 08:04:36.278401 13661 consensus_queue.cc:260] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.279467 13665 tablet_bootstrap.cc:492] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.280694 13661 ts_tablet_manager.cc:1398] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240503 08:04:36.281638 13661 tablet_bootstrap.cc:492] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:36.276517 13660 ts_tablet_manager.cc:1398] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20240503 08:04:36.285435 13665 tablet_bootstrap.cc:654] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.275933 13675 raft_consensus.cc:504] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.285959 13660 tablet_bootstrap.cc:492] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:36.287371 13675 leader_election.cc:288] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.293640 13661 tablet_bootstrap.cc:654] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.304476 13959 raft_consensus.cc:482] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.305028 13959 raft_consensus.cc:504] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.309273 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:36.309996 13543 raft_consensus.cc:2454] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:36.311436 13583 leader_election.cc:302] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:36.312245 13850 raft_consensus.cc:2781] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.312577 13850 raft_consensus.cc:482] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.312877 13850 raft_consensus.cc:3037] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.315099 13660 tablet_bootstrap.cc:654] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.320412 13661 tablet_bootstrap.cc:492] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:36.320873 13661 ts_tablet_manager.cc:1367] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.000s
I20240503 08:04:36.323797 13661 raft_consensus.cc:348] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.324754 13661 raft_consensus.cc:374] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.325114 13661 raft_consensus.cc:729] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.325942 13661 consensus_queue.cc:260] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.326759 13675 raft_consensus.cc:482] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.327445 13675 raft_consensus.cc:504] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.330121 13661 ts_tablet_manager.cc:1398] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20240503 08:04:36.307042 13959 leader_election.cc:288] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:36.332156 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "878ffcf68a1b4726a683f6590d94704c" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.332916 13620 raft_consensus.cc:2379] T 878ffcf68a1b4726a683f6590d94704c P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:36.333709 13660 tablet_bootstrap.cc:492] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:36.334141 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "878ffcf68a1b4726a683f6590d94704c" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.334264 13660 ts_tablet_manager.cc:1367] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.048s user 0.021s sys 0.002s
I20240503 08:04:36.334174 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.334964 13466 raft_consensus.cc:2379] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:36.333593 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.335085 13468 raft_consensus.cc:2454] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:36.336177 13661 tablet_bootstrap.cc:492] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:36.336889 13850 raft_consensus.cc:504] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.338140 13618 raft_consensus.cc:2379] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
I20240503 08:04:36.337963 13660 raft_consensus.cc:348] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.338694 13660 raft_consensus.cc:374] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.338898 13850 leader_election.cc:288] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:36.339005 13660 raft_consensus.cc:729] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
W20240503 08:04:36.340036 13437 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 62356.1us
I20240503 08:04:36.339774 13660 consensus_queue.cc:260] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.342149 13511 leader_election.cc:302] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.342989 13927 raft_consensus.cc:2781] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.343492 13927 raft_consensus.cc:482] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.343935 13927 raft_consensus.cc:3037] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.345024 13512 leader_election.cc:302] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:36.346017 13752 raft_consensus.cc:3037] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.347527 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:36.351440 13843 raft_consensus.cc:482] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.352007 13843 raft_consensus.cc:504] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.352941 13752 raft_consensus.cc:2726] T 878ffcf68a1b4726a683f6590d94704c P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:36.355286 13843 leader_election.cc:288] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.351421 13927 raft_consensus.cc:504] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.356688 13437 leader_election.cc:302] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:36.360421 13543 raft_consensus.cc:1261] T e833906e87754702bba4e5dbde972f86 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 2 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240503 08:04:36.356807 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.357276 13675 leader_election.cc:288] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.358753 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4061d55705594102a8b057896c5f86d0" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.365096 13959 consensus_queue.cc:1035] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:36.365691 13512 leader_election.cc:302] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:36.366763 13675 raft_consensus.cc:2726] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:36.357782 13661 tablet_bootstrap.cc:654] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.360494 13748 raft_consensus.cc:2781] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.377835 13748 raft_consensus.cc:686] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:36.378747 13748 consensus_queue.cc:237] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.381489 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.382189 13466 raft_consensus.cc:2454] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:36.359373 13660 ts_tablet_manager.cc:1398] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.025s user 0.009s sys 0.000s
I20240503 08:04:36.385896 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.386579 13471 raft_consensus.cc:3037] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.385723 13660 tablet_bootstrap.cc:492] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:36.404026 13544 raft_consensus.cc:2379] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:36.412663 13471 raft_consensus.cc:2454] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:36.412969 13689 consensus_queue.cc:1035] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:36.422885 13665 tablet_bootstrap.cc:492] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.423401 13665 ts_tablet_manager.cc:1367] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.144s user 0.043s sys 0.048s
I20240503 08:04:36.425196 13585 leader_election.cc:302] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7
I20240503 08:04:36.425984 13665 raft_consensus.cc:348] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.426723 13665 raft_consensus.cc:374] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.426985 13665 raft_consensus.cc:729] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.427659 13665 consensus_queue.cc:260] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.429710 13665 ts_tablet_manager.cc:1398] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:36.429986 13762 raft_consensus.cc:2781] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.430567 13665 tablet_bootstrap.cc:492] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.430694 13762 raft_consensus.cc:686] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:36.431897 13762 consensus_queue.cc:237] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.437845 13661 tablet_bootstrap.cc:492] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:36.438148 13689 raft_consensus.cc:482] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.439040 13665 tablet_bootstrap.cc:654] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.438999 13689 raft_consensus.cc:504] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.441704 13689 leader_election.cc:288] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.438532 13661 ts_tablet_manager.cc:1367] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.103s user 0.022s sys 0.005s
I20240503 08:04:36.445843 13661 raft_consensus.cc:348] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.445659 13368 catalog_manager.cc:5484] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:36.446520 13661 raft_consensus.cc:374] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.446821 13661 raft_consensus.cc:729] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.447504 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969c8edb68343339904e50c63bbdace" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.448253 13619 raft_consensus.cc:2454] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.450624 13437 leader_election.cc:302] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.450367 13661 consensus_queue.cc:260] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.453030 13661 ts_tablet_manager.cc:1398] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20240503 08:04:36.454025 13748 raft_consensus.cc:2781] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.454236 13661 tablet_bootstrap.cc:492] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:36.456050 13748 raft_consensus.cc:482] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.456421 13748 raft_consensus.cc:3037] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.462713 13661 tablet_bootstrap.cc:654] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.470624 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.414271 13927 leader_election.cc:288] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.414791 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691511776999428eae4eee8e73de08ce" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.472704 13468 raft_consensus.cc:2379] T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate cf70d9fd04894ea3812767cdb12156c5 in this term.
I20240503 08:04:36.479512 13927 raft_consensus.cc:482] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.480401 13927 raft_consensus.cc:504] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.487315 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0b468643254b1dba30573778f90be8" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.487661 13543 raft_consensus.cc:3037] T 4061d55705594102a8b057896c5f86d0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.488071 13468 raft_consensus.cc:2454] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:36.418131 13660 tablet_bootstrap.cc:654] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.489389 13511 leader_election.cc:302] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.490319 13752 raft_consensus.cc:2781] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.490867 13752 raft_consensus.cc:482] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.491221 13752 raft_consensus.cc:3037] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.509069 13752 raft_consensus.cc:504] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.514204 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0b468643254b1dba30573778f90be8" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.514853 13468 raft_consensus.cc:3037] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.518724 13661 tablet_bootstrap.cc:492] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:36.519488 13661 ts_tablet_manager.cc:1367] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.008s
I20240503 08:04:36.521922 13661 raft_consensus.cc:348] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.525323 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969c8edb68343339904e50c63bbdace" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:36.526075 13689 raft_consensus.cc:482] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.526804 13468 raft_consensus.cc:2454] T fe0b468643254b1dba30573778f90be8 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:36.526600 13541 raft_consensus.cc:2454] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.526648 13689 raft_consensus.cc:504] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.528219 13661 raft_consensus.cc:374] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.528096 13511 leader_election.cc:302] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.529310 13661 raft_consensus.cc:729] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.531991 13752 leader_election.cc:288] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.535672 13661 consensus_queue.cc:260] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.535679 13748 raft_consensus.cc:504] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.538169 13748 leader_election.cc:288] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.540174 13661 ts_tablet_manager.cc:1398] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.020s user 0.004s sys 0.002s
I20240503 08:04:36.539988 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969c8edb68343339904e50c63bbdace" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.541915 13619 raft_consensus.cc:3037] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.541076 13665 tablet_bootstrap.cc:492] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.542899 13665 ts_tablet_manager.cc:1367] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.113s user 0.041s sys 0.042s
I20240503 08:04:36.546465 13689 leader_election.cc:288] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.547717 13843 raft_consensus.cc:482] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.547998 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "008315f470c24029bd9271aad3c67a09" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.548240 13843 raft_consensus.cc:504] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.546635 13619 raft_consensus.cc:2454] T 6969c8edb68343339904e50c63bbdace P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:36.548812 13368 catalog_manager.cc:5484] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:36.548853 13617 raft_consensus.cc:2454] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.550122 13437 leader_election.cc:302] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.550284 13843 leader_election.cc:288] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.550851 13843 raft_consensus.cc:2781] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.551306 13843 raft_consensus.cc:686] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:36.552646 13665 raft_consensus.cc:348] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.551280 13437 leader_election.cc:302] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.553421 13665 raft_consensus.cc:374] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.553778 13665 raft_consensus.cc:729] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.552513 13843 consensus_queue.cc:237] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.554476 13665 consensus_queue.cc:260] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.558161 13665 ts_tablet_manager.cc:1398] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20240503 08:04:36.560787 13665 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.564625 13850 raft_consensus.cc:482] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.565261 13927 leader_election.cc:288] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.565200 13850 raft_consensus.cc:504] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.579085 13850 leader_election.cc:288] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:36.564813 13689 raft_consensus.cc:2781] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.566166 13754 raft_consensus.cc:2781] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.580472 13754 raft_consensus.cc:482] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.580874 13754 raft_consensus.cc:3037] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.580570 13689 raft_consensus.cc:686] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:36.582607 13689 consensus_queue.cc:237] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.584095 13471 raft_consensus.cc:1261] T e833906e87754702bba4e5dbde972f86 P f2462cc996fb441a847521d550b300a4 [term 2 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240503 08:04:36.585399 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e49c587b8a40e7a6eee4ee6566de1e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.585530 13927 consensus_queue.cc:1035] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:36.586134 13466 raft_consensus.cc:2454] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:36.586972 13464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.587507 13754 raft_consensus.cc:504] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.589591 13754 leader_election.cc:288] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.590889 13511 leader_election.cc:302] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
W20240503 08:04:36.593876 13585 leader_election.cc:341] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:36.594928 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "008315f470c24029bd9271aad3c67a09" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.595603 13619 raft_consensus.cc:3037] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.567119 13665 tablet_bootstrap.cc:654] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.597757 13999 consensus_queue.cc:1035] T e833906e87754702bba4e5dbde972f86 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:36.599442 13927 raft_consensus.cc:2781] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.600963 13748 raft_consensus.cc:482] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.601692 13748 raft_consensus.cc:504] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.600003 13927 raft_consensus.cc:482] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.604766 13927 raft_consensus.cc:3037] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.609182 13619 raft_consensus.cc:2454] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:36.609829 13748 leader_election.cc:288] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.622922 13661 tablet_bootstrap.cc:492] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:36.624219 13665 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.624701 13665 ts_tablet_manager.cc:1367] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.064s user 0.010s sys 0.005s
I20240503 08:04:36.630999 13665 raft_consensus.cc:348] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.631824 13665 raft_consensus.cc:374] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.632624 13665 raft_consensus.cc:729] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.633226 13665 consensus_queue.cc:260] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.636397 13665 ts_tablet_manager.cc:1398] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20240503 08:04:36.639719 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.640473 13619 raft_consensus.cc:2454] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.640955 13665 tablet_bootstrap.cc:492] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.642221 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "008315f470c24029bd9271aad3c67a09" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:36.639719 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6969c8edb68343339904e50c63bbdace" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:36.639305 13927 raft_consensus.cc:504] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.643860 13544 raft_consensus.cc:3037] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.644194 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "008315f470c24029bd9271aad3c67a09" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:36.654186 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e49c587b8a40e7a6eee4ee6566de1e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.654852 13465 raft_consensus.cc:3037] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.648808 13927 leader_election.cc:288] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.642917 13541 raft_consensus.cc:2454] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.646314 13437 leader_election.cc:302] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.657516 13754 raft_consensus.cc:2781] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.657941 13754 raft_consensus.cc:686] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:36.658705 13544 raft_consensus.cc:2454] T 6969c8edb68343339904e50c63bbdace P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:36.658663 13754 consensus_queue.cc:237] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.661242 13437 leader_election.cc:302] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.666306 13665 tablet_bootstrap.cc:654] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.667924 13927 raft_consensus.cc:482] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.662024 13748 raft_consensus.cc:2781] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.668731 13748 raft_consensus.cc:482] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.667981 13367 catalog_manager.cc:5484] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:36.668604 13927 raft_consensus.cc:504] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.669102 13661 tablet_bootstrap.cc:654] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:36.669293 13748 raft_consensus.cc:3037] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.673974 13927 leader_election.cc:288] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.677322 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0b468643254b1dba30573778f90be8" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.672667 13369 catalog_manager.cc:5484] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:36.687510 13512 leader_election.cc:302] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:36.701130 13752 raft_consensus.cc:2726] T 691511776999428eae4eee8e73de08ce P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 08:04:36.701026 14001 raft_consensus.cc:482] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.704444 14001 raft_consensus.cc:504] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.689743 13464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.716046 13660 tablet_bootstrap.cc:492] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:36.717293 13660 ts_tablet_manager.cc:1367] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.332s user 0.032s sys 0.014s
I20240503 08:04:36.701644 13542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:36.709125 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e49c587b8a40e7a6eee4ee6566de1e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.733789 13748 raft_consensus.cc:504] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.738365 13748 leader_election.cc:288] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.739993 13620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.740538 13620 raft_consensus.cc:3037] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.726205 13542 raft_consensus.cc:2454] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:36.745911 13689 consensus_queue.cc:1035] T 5e13da943b8b4caa8da56b29cd48f93f P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:36.746039 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0b468643254b1dba30573778f90be8" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.742352 14001 leader_election.cc:288] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.764431 13661 tablet_bootstrap.cc:492] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:36.765215 13661 ts_tablet_manager.cc:1367] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.143s user 0.015s sys 0.009s
I20240503 08:04:36.747833 13464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:36.750059 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.750845 13616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e49c587b8a40e7a6eee4ee6566de1e" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:36.753350 13660 raft_consensus.cc:348] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:36.754678 13511 leader_election.cc:341] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:36.769701 13620 raft_consensus.cc:2454] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:36.771644 13661 raft_consensus.cc:348] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.775854 13660 raft_consensus.cc:374] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.776350 13661 raft_consensus.cc:374] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.776695 13660 raft_consensus.cc:729] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.776710 13661 raft_consensus.cc:729] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.777966 13437 leader_election.cc:302] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:36.777491 13660 consensus_queue.cc:260] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.775671 13614 raft_consensus.cc:2454] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:36.779546 13617 raft_consensus.cc:2454] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:36.780246 13465 raft_consensus.cc:2454] T 88e49c587b8a40e7a6eee4ee6566de1e P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:36.781316 13660 ts_tablet_manager.cc:1398] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.064s user 0.004s sys 0.002s
I20240503 08:04:36.777493 13661 consensus_queue.cc:260] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.775816 13464 raft_consensus.cc:2379] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:36.809398 14013 raft_consensus.cc:2781] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.816754 13660 tablet_bootstrap.cc:492] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:36.817636 13661 ts_tablet_manager.cc:1398] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.052s user 0.009s sys 0.000s
I20240503 08:04:36.818529 13661 tablet_bootstrap.cc:492] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:36.819734 14013 raft_consensus.cc:686] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:36.825186 14013 consensus_queue.cc:237] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:36.827067 13512 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 119334us
W20240503 08:04:36.828047 13512 leader_election.cc:341] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:36.828851 13512 leader_election.cc:341] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
W20240503 08:04:36.830509 13512 leader_election.cc:341] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:36.831156 13512 leader_election.cc:302] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:36.835309 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:36.836989 13434 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 93574.1us
I20240503 08:04:36.836182 13675 raft_consensus.cc:2781] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:36.839107 13675 raft_consensus.cc:482] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:36.839490 13675 raft_consensus.cc:3037] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.832718 13927 consensus_queue.cc:1035] T aa42fc3b9d894d398b7002af94aa3f40 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20240503 08:04:36.860613 13748 consensus_queue.cc:1035] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:36.864784 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:36.865496 13538 raft_consensus.cc:3037] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:36.870108 13511 leader_election.cc:302] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:36.873378 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e79e29310743ff852e4dfe1d8da13f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:36.874274 13614 raft_consensus.cc:2379] T c2e79e29310743ff852e4dfe1d8da13f P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in current term 1: Already voted for candidate f2462cc996fb441a847521d550b300a4 in this term.
I20240503 08:04:36.878510 14010 raft_consensus.cc:2781] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:36.879156 14010 raft_consensus.cc:686] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:36.880013 14010 consensus_queue.cc:237] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.912400 13512 leader_election.cc:302] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:36.925462 13665 tablet_bootstrap.cc:492] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:36.926174 13665 ts_tablet_manager.cc:1367] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.286s user 0.034s sys 0.062s
I20240503 08:04:36.948390 13665 raft_consensus.cc:348] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.949188 13665 raft_consensus.cc:374] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:36.949445 13665 raft_consensus.cc:729] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:36.950030 13665 consensus_queue.cc:260] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.929868 13675 raft_consensus.cc:504] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:36.957947 13544 raft_consensus.cc:2379] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf70d9fd04894ea3812767cdb12156c5 in current term 1: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:36.959962 13675 leader_election.cc:288] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:36.964176 13466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:36.996866 13689 raft_consensus.cc:482] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:36.997992 13665 ts_tablet_manager.cc:1398] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.071s user 0.005s sys 0.002s
W20240503 08:04:36.973538 13511 leader_election.cc:341] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:37.003006 13689 raft_consensus.cc:504] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.003835 13665 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:36.972599 13661 tablet_bootstrap.cc:654] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:37.004506 13512 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 74800.2us
I20240503 08:04:37.001222 13538 raft_consensus.cc:2454] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:37.006417 13660 tablet_bootstrap.cc:654] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.018564 13689 leader_election.cc:288] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:37.014389 13843 raft_consensus.cc:2726] T c2e79e29310743ff852e4dfe1d8da13f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:37.026827 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:37.033535 14016 consensus_queue.cc:1035] T 878ffcf68a1b4726a683f6590d94704c P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20240503 08:04:37.034550 13617 raft_consensus.cc:3037] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.044504 13539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2966873f74257a9461592adbf7573" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:37.045341 13539 raft_consensus.cc:2454] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:37.046284 13613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2966873f74257a9461592adbf7573" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:37.050539 13762 consensus_queue.cc:1035] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20240503 08:04:37.056318 13437 leader_election.cc:341] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:37.047356 14016 consensus_queue.cc:1035] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.060612 13434 leader_election.cc:302] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters: cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:37.062392 13861 raft_consensus.cc:2781] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:37.062892 13861 raft_consensus.cc:482] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:37.063323 13861 raft_consensus.cc:3037] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.101462 13367 catalog_manager.cc:5484] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:37.095572 13368 catalog_manager.cc:5484] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:37.104146 13367 catalog_manager.cc:5484] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:37.115702 13617 raft_consensus.cc:2454] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:37.127007 13512 leader_election.cc:302] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:37.145591 13752 raft_consensus.cc:2781] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:37.165482 13583 leader_election.cc:302] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:37.177228 13762 raft_consensus.cc:2726] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:37.195216 13660 tablet_bootstrap.cc:492] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:37.195698 13660 ts_tablet_manager.cc:1367] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.379s user 0.053s sys 0.050s
I20240503 08:04:37.198180 13660 raft_consensus.cc:348] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.198887 13660 raft_consensus.cc:374] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.199196 13660 raft_consensus.cc:729] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.199419 13861 raft_consensus.cc:504] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.199831 13660 consensus_queue.cc:260] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.201462 13660 ts_tablet_manager.cc:1398] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 08:04:37.202699 13660 tablet_bootstrap.cc:492] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:37.203884 13752 raft_consensus.cc:686] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:37.204939 13752 consensus_queue.cc:237] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.207299 13661 tablet_bootstrap.cc:492] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:37.207824 13661 ts_tablet_manager.cc:1367] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.389s user 0.036s sys 0.019s
I20240503 08:04:37.208348 13660 tablet_bootstrap.cc:654] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:37.207737 13511 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.1:34427, user_credentials={real_user=slave}} blocked reactor thread for 48323.6us
I20240503 08:04:37.211781 14023 consensus_queue.cc:1035] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.210299 13661 raft_consensus.cc:348] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.213661 13661 raft_consensus.cc:374] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.214030 13661 raft_consensus.cc:729] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.215426 14016 consensus_queue.cc:1035] T 6969c8edb68343339904e50c63bbdace P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:37.214633 13661 consensus_queue.cc:260] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.217326 13661 ts_tablet_manager.cc:1398] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20240503 08:04:37.218338 13661 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:37.219789 13660 tablet_bootstrap.cc:492] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:37.219188 14014 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.220265 13660 ts_tablet_manager.cc:1367] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20240503 08:04:37.222651 13660 raft_consensus.cc:348] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.223407 13660 raft_consensus.cc:374] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.223693 13660 raft_consensus.cc:729] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
W20240503 08:04:37.224454 14028 log.cc:924] Time spent T 691511776999428eae4eee8e73de08ce P f2462cc996fb441a847521d550b300a4: Append to log took a long time: real 0.106s user 0.000s sys 0.000s
I20240503 08:04:37.224618 13660 consensus_queue.cc:260] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.226270 13660 ts_tablet_manager.cc:1398] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240503 08:04:37.227090 13660 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
W20240503 08:04:37.228590 13512 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 101677us
I20240503 08:04:37.230420 13616 raft_consensus.cc:3037] T fe0b468643254b1dba30573778f90be8 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.230609 13861 leader_election.cc:288] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:37.234738 13618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2966873f74257a9461592adbf7573" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:37.203014 13665 tablet_bootstrap.cc:654] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.237640 13618 raft_consensus.cc:3037] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.272528 13618 raft_consensus.cc:2454] T 98a2966873f74257a9461592adbf7573 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
W20240503 08:04:37.289799 13437 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.3:33827, user_credentials={real_user=slave}} blocked reactor thread for 45475us
I20240503 08:04:37.293097 13539 raft_consensus.cc:3037] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.296377 13661 tablet_bootstrap.cc:654] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
W20240503 08:04:37.291961 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:37.330792 13437 leader_election.cc:302] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:37.340535 13544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2966873f74257a9461592adbf7573" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:37.341449 13544 raft_consensus.cc:3037] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:37.345058 13434 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 71775.1us
I20240503 08:04:37.345979 13748 raft_consensus.cc:2781] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:37.346419 13748 raft_consensus.cc:686] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:37.347162 13748 consensus_queue.cc:237] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:37.329962 13583 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 43940us
W20240503 08:04:37.363287 13512 consensus_peers.cc:485] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 -> Peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Couldn't send request to peer cf70d9fd04894ea3812767cdb12156c5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:37.364943 13661 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:37.365456 13661 ts_tablet_manager.cc:1367] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.147s user 0.015s sys 0.003s
I20240503 08:04:37.380002 13544 raft_consensus.cc:2454] T 98a2966873f74257a9461592adbf7573 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:37.382086 13660 tablet_bootstrap.cc:654] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.392997 13665 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:37.393507 13665 ts_tablet_manager.cc:1367] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.390s user 0.009s sys 0.052s
I20240503 08:04:37.388850 13661 raft_consensus.cc:348] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.396490 13661 raft_consensus.cc:374] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.396821 13661 raft_consensus.cc:729] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.411793 13661 consensus_queue.cc:260] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.437479 13661 ts_tablet_manager.cc:1398] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.072s user 0.003s sys 0.015s
I20240503 08:04:37.437651 13665 raft_consensus.cc:348] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.438325 13665 raft_consensus.cc:374] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.438481 13661 tablet_bootstrap.cc:492] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:37.438593 13665 raft_consensus.cc:729] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.439500 13665 consensus_queue.cc:260] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.442121 13927 consensus_queue.cc:1035] T fe0b468643254b1dba30573778f90be8 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 08:04:37.426360 13583 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 62279.7us
I20240503 08:04:37.452148 13665 ts_tablet_manager.cc:1398] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.058s user 0.005s sys 0.005s
I20240503 08:04:37.453454 13665 tablet_bootstrap.cc:492] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:37.460748 13661 tablet_bootstrap.cc:654] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.470753 14042 consensus_queue.cc:1035] T 691511776999428eae4eee8e73de08ce P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.481130 13665 tablet_bootstrap.cc:654] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.448486 13927 consensus_queue.cc:1035] T 88e49c587b8a40e7a6eee4ee6566de1e P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20240503 08:04:37.512264 13367 catalog_manager.cc:5484] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:37.531041 13660 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:37.531741 13660 ts_tablet_manager.cc:1367] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.305s user 0.018s sys 0.027s
I20240503 08:04:37.534850 13660 raft_consensus.cc:348] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.535792 13660 raft_consensus.cc:374] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.536119 13660 raft_consensus.cc:729] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.536929 13660 consensus_queue.cc:260] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
W20240503 08:04:37.526475 13583 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 78168.2us
I20240503 08:04:37.579705 13660 ts_tablet_manager.cc:1398] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.048s user 0.008s sys 0.002s
I20240503 08:04:37.581079 13660 tablet_bootstrap.cc:492] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:37.552430 14013 raft_consensus.cc:482] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:37.593684 13748 raft_consensus.cc:482] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:37.596731 13368 catalog_manager.cc:5484] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:37.594228 13748 raft_consensus.cc:504] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.618050 13748 leader_election.cc:288] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:37.629281 13613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:37.594992 14013 raft_consensus.cc:504] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.644364 13964 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20240503 08:04:37.648233 14014 consensus_queue.cc:1035] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.656953 13661 tablet_bootstrap.cc:492] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:37.657383 13661 ts_tablet_manager.cc:1367] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.219s user 0.019s sys 0.013s
I20240503 08:04:37.661895 13660 tablet_bootstrap.cc:654] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.654989 13613 raft_consensus.cc:2379] T f166d4169b16438fa99c721ab4c393ff P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 1: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:37.673504 13999 raft_consensus.cc:482] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:37.676571 13999 raft_consensus.cc:504] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.677146 14013 leader_election.cc:288] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:37.679313 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f166d4169b16438fa99c721ab4c393ff" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:37.651245 14016 consensus_queue.cc:1035] T c2e79e29310743ff852e4dfe1d8da13f P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20240503 08:04:37.689062 13613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef1acee9ffb4310bde01caa87d35754" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:37.689790 13613 raft_consensus.cc:2454] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:37.691258 13539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e689a2dc7941258f45f121e31c75bd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:37.701459 13583 leader_election.cc:341] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:37.703610 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef1acee9ffb4310bde01caa87d35754" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:37.704388 13543 raft_consensus.cc:2454] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:37.678530 13999 leader_election.cc:288] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:37.723083 13437 leader_election.cc:302] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:37.735838 13434 leader_election.cc:302] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:37.739423 13754 raft_consensus.cc:2781] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:37.740274 13754 raft_consensus.cc:482] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:37.740901 13754 raft_consensus.cc:3037] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.738659 13861 raft_consensus.cc:3037] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.739221 13468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e689a2dc7941258f45f121e31c75bd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:37.758349 13661 raft_consensus.cc:348] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.759243 13661 raft_consensus.cc:374] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.759589 13661 raft_consensus.cc:729] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.757457 13754 raft_consensus.cc:504] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.760397 13661 consensus_queue.cc:260] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.766059 13754 leader_election.cc:288] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
W20240503 08:04:37.771550 13585 leader_election.cc:341] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:37.772043 13585 leader_election.cc:302] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:37.774559 14044 raft_consensus.cc:2726] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:37.788661 13754 consensus_queue.cc:1035] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:37.814467 13545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef1acee9ffb4310bde01caa87d35754" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:37.831691 13661 ts_tablet_manager.cc:1398] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.174s user 0.007s sys 0.010s
I20240503 08:04:37.840044 13661 tablet_bootstrap.cc:492] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
W20240503 08:04:37.833490 13434 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.2:44917, user_credentials={real_user=slave}} blocked reactor thread for 44314.9us
I20240503 08:04:37.851269 13660 tablet_bootstrap.cc:492] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:37.851771 13660 ts_tablet_manager.cc:1367] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.271s user 0.010s sys 0.012s
I20240503 08:04:37.829342 13545 raft_consensus.cc:3037] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.827692 13616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef1acee9ffb4310bde01caa87d35754" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:37.856153 13616 raft_consensus.cc:3037] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:37.866201 13661 tablet_bootstrap.cc:654] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.866475 13665 tablet_bootstrap.cc:492] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:37.866955 13665 ts_tablet_manager.cc:1367] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.414s user 0.020s sys 0.020s
I20240503 08:04:37.870402 14023 consensus_queue.cc:1035] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.873359 13660 raft_consensus.cc:348] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.874409 13660 raft_consensus.cc:374] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.874703 13660 raft_consensus.cc:729] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.875486 13660 consensus_queue.cc:260] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.885432 13660 ts_tablet_manager.cc:1398] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20240503 08:04:37.886338 13660 tablet_bootstrap.cc:492] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:37.875598 13861 raft_consensus.cc:2726] T f166d4169b16438fa99c721ab4c393ff P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:37.905164 14001 consensus_queue.cc:1035] T f166d4169b16438fa99c721ab4c393ff P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:37.909708 13545 raft_consensus.cc:2454] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:37.921615 13665 raft_consensus.cc:348] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.922133 13434 leader_election.cc:302] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:37.922459 13665 raft_consensus.cc:374] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.922966 13665 raft_consensus.cc:729] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.923350 13689 raft_consensus.cc:2781] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:37.923877 13689 raft_consensus.cc:686] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:37.923619 13665 consensus_queue.cc:260] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.924751 13689 consensus_queue.cc:237] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.926126 13665 ts_tablet_manager.cc:1398] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.052s user 0.006s sys 0.001s
I20240503 08:04:37.927047 13665 tablet_bootstrap.cc:492] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:37.933442 13665 tablet_bootstrap.cc:654] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.919500 13616 raft_consensus.cc:2454] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:37.948335 13660 tablet_bootstrap.cc:654] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:37.953655 13369 catalog_manager.cc:5484] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:37.921223 13861 consensus_queue.cc:1035] T 98a2966873f74257a9461592adbf7573 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20240503 08:04:37.973819 13665 tablet_bootstrap.cc:492] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:37.974301 13665 ts_tablet_manager.cc:1367] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.005s
I20240503 08:04:37.976641 13665 raft_consensus.cc:348] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.977598 13665 raft_consensus.cc:374] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:37.977969 13665 raft_consensus.cc:729] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:37.978860 13665 consensus_queue.cc:260] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:37.992105 13660 tablet_bootstrap.cc:492] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:37.992692 13660 ts_tablet_manager.cc:1367] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.107s user 0.013s sys 0.006s
I20240503 08:04:37.997428 13665 ts_tablet_manager.cc:1398] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.023s user 0.009s sys 0.001s
I20240503 08:04:37.998445 13665 tablet_bootstrap.cc:492] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:38.002722 13660 raft_consensus.cc:348] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.003623 13660 raft_consensus.cc:374] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.003968 13660 raft_consensus.cc:729] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.004703 13660 consensus_queue.cc:260] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.007462 13660 ts_tablet_manager.cc:1398] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20240503 08:04:38.009189 13660 tablet_bootstrap.cc:492] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:38.020970 13661 tablet_bootstrap.cc:492] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:38.021457 13665 tablet_bootstrap.cc:654] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.021566 13661 ts_tablet_manager.cc:1367] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.182s user 0.019s sys 0.033s
I20240503 08:04:38.024302 13661 raft_consensus.cc:348] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.025143 13661 raft_consensus.cc:374] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.021771 13660 tablet_bootstrap.cc:654] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.028688 13661 raft_consensus.cc:729] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.030100 13661 consensus_queue.cc:260] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.031011 13665 tablet_bootstrap.cc:492] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:38.051985 13665 ts_tablet_manager.cc:1367] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.006s
I20240503 08:04:38.054455 13665 raft_consensus.cc:348] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.032665 13661 ts_tablet_manager.cc:1398] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20240503 08:04:38.059613 13661 tablet_bootstrap.cc:492] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:38.065879 13665 raft_consensus.cc:374] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.066246 13665 raft_consensus.cc:729] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.067564 13665 consensus_queue.cc:260] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.073645 13665 ts_tablet_manager.cc:1398] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.021s user 0.004s sys 0.003s
I20240503 08:04:38.074607 13665 tablet_bootstrap.cc:492] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:38.075515 13660 tablet_bootstrap.cc:492] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:38.075973 13660 ts_tablet_manager.cc:1367] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.067s user 0.018s sys 0.018s
I20240503 08:04:38.078455 13660 raft_consensus.cc:348] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.079206 13660 raft_consensus.cc:374] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.079509 13660 raft_consensus.cc:729] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.080097 13660 consensus_queue.cc:260] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.084071 13660 ts_tablet_manager.cc:1398] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240503 08:04:38.085428 13660 tablet_bootstrap.cc:492] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:38.089128 13665 tablet_bootstrap.cc:654] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.142954 13661 tablet_bootstrap.cc:654] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.144268 14023 raft_consensus.cc:482] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.149257 14017 raft_consensus.cc:482] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.146047 14023 raft_consensus.cc:504] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.150305 14017 raft_consensus.cc:504] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.196835 13660 tablet_bootstrap.cc:700] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5: Time spent opening tablet: real 0.104s user 0.002s sys 0.000s
I20240503 08:04:38.198042 14017 leader_election.cc:288] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.221618 13616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556938a7848d410b8ee50c324d57228f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:38.216805 14023 leader_election.cc:288] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.225898 13616 raft_consensus.cc:2454] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:38.234809 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e689a2dc7941258f45f121e31c75bd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:38.235798 13614 raft_consensus.cc:2454] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:38.229452 13927 raft_consensus.cc:482] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.238478 13927 raft_consensus.cc:504] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.238860 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556938a7848d410b8ee50c324d57228f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:38.241415 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e689a2dc7941258f45f121e31c75bd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:38.242558 13538 raft_consensus.cc:2454] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 0.
I20240503 08:04:38.243721 13762 raft_consensus.cc:482] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.247378 13762 raft_consensus.cc:504] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.248759 13660 tablet_bootstrap.cc:654] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.250049 13762 leader_election.cc:288] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.251704 13927 leader_election.cc:288] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.253831 13512 leader_election.cc:302] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:38.255842 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e49c587b8a40e7a6eee4ee6566de1e" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:38.256390 13437 leader_election.cc:302] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:38.258405 14052 raft_consensus.cc:2781] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:38.258908 14052 raft_consensus.cc:482] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:38.259310 14052 raft_consensus.cc:3037] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
W20240503 08:04:38.261606 13511 leader_election.cc:341] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:38.263589 13665 tablet_bootstrap.cc:492] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:38.264420 13665 ts_tablet_manager.cc:1367] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.190s user 0.005s sys 0.011s
I20240503 08:04:38.266652 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13577f32ef6f412fb64c5ae26ec2f13b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:38.267527 13927 raft_consensus.cc:2781] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:38.268015 13927 raft_consensus.cc:482] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:38.271521 13927 raft_consensus.cc:3037] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.275600 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13577f32ef6f412fb64c5ae26ec2f13b" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
W20240503 08:04:38.282305 13512 leader_election.cc:341] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:38.282552 13540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e49c587b8a40e7a6eee4ee6566de1e" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:38.288939 13511 leader_election.cc:341] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:38.289029 13583 leader_election.cc:302] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4
I20240503 08:04:38.289564 13511 leader_election.cc:302] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7; no voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4
I20240503 08:04:38.290045 13999 raft_consensus.cc:2726] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:38.291086 14049 raft_consensus.cc:2726] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:38.311600 13999 raft_consensus.cc:482] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.311862 13665 raft_consensus.cc:348] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.312737 13665 raft_consensus.cc:374] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.313114 13665 raft_consensus.cc:729] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.313978 13665 consensus_queue.cc:260] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.312346 13999 raft_consensus.cc:504] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.317603 13999 leader_election.cc:288] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.324889 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4999c55f4034c3d9997841353a35afd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:38.325767 13465 raft_consensus.cc:2454] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:38.327064 13585 leader_election.cc:302] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:38.327971 13999 raft_consensus.cc:2781] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:38.328370 13999 raft_consensus.cc:482] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:38.328761 13999 raft_consensus.cc:3037] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.328902 14052 raft_consensus.cc:504] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.331360 13661 tablet_bootstrap.cc:492] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:38.332121 13661 ts_tablet_manager.cc:1367] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.273s user 0.007s sys 0.010s
I20240503 08:04:38.336326 13665 ts_tablet_manager.cc:1398] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.072s user 0.006s sys 0.004s
I20240503 08:04:38.339589 13999 raft_consensus.cc:504] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.343238 13999 leader_election.cc:288] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.347261 13665 tablet_bootstrap.cc:492] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:38.348524 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4999c55f4034c3d9997841353a35afd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:38.349294 13465 raft_consensus.cc:3037] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.353003 13927 raft_consensus.cc:504] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.353261 13661 raft_consensus.cc:348] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.354013 13661 raft_consensus.cc:374] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.354293 13661 raft_consensus.cc:729] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.359509 13661 consensus_queue.cc:260] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.373881 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556938a7848d410b8ee50c324d57228f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
W20240503 08:04:38.376488 13511 leader_election.cc:341] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:38.378818 13540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4999c55f4034c3d9997841353a35afd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:38.379603 13540 raft_consensus.cc:3037] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.379527 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4999c55f4034c3d9997841353a35afd" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:38.380137 13660 tablet_bootstrap.cc:492] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:38.380736 13660 ts_tablet_manager.cc:1367] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.296s user 0.013s sys 0.014s
I20240503 08:04:38.382015 13661 ts_tablet_manager.cc:1398] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.050s user 0.008s sys 0.001s
I20240503 08:04:38.383277 13661 tablet_bootstrap.cc:492] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:38.390810 13465 raft_consensus.cc:2454] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:38.390967 13540 raft_consensus.cc:2454] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:38.391894 13585 leader_election.cc:302] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:38.392674 13999 raft_consensus.cc:2781] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:38.393163 13999 raft_consensus.cc:686] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:38.394232 13999 consensus_queue.cc:237] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.395912 14017 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:38.399734 14052 leader_election.cc:288] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.400447 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e689a2dc7941258f45f121e31c75bd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:38.401140 13543 raft_consensus.cc:3037] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.403038 13665 tablet_bootstrap.cc:654] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.407866 13617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e689a2dc7941258f45f121e31c75bd" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:38.408226 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556938a7848d410b8ee50c324d57228f" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:38.408603 13617 raft_consensus.cc:3037] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.408845 13614 raft_consensus.cc:3037] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.412180 13661 tablet_bootstrap.cc:654] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.413385 13927 leader_election.cc:288] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.418377 13614 raft_consensus.cc:2454] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:38.421295 13660 raft_consensus.cc:348] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.422060 13660 raft_consensus.cc:374] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.422328 13660 raft_consensus.cc:729] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.422920 13660 consensus_queue.cc:260] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.428118 13543 raft_consensus.cc:2454] T 41e689a2dc7941258f45f121e31c75bd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:38.429497 13660 ts_tablet_manager.cc:1398] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.048s user 0.008s sys 0.000s
I20240503 08:04:38.436897 13660 tablet_bootstrap.cc:492] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:38.442021 13512 leader_election.cc:302] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters: f2462cc996fb441a847521d550b300a4
I20240503 08:04:38.443349 14049 raft_consensus.cc:2781] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:38.444944 13434 leader_election.cc:302] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:38.443924 14049 raft_consensus.cc:686] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:38.446172 14049 consensus_queue.cc:237] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.455058 14014 raft_consensus.cc:2781] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:38.463935 14014 raft_consensus.cc:686] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Becoming Leader. State: Replica: f2462cc996fb441a847521d550b300a4, State: Running, Role: LEADER
I20240503 08:04:38.453944 13617 raft_consensus.cc:2454] T 41e689a2dc7941258f45f121e31c75bd P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2462cc996fb441a847521d550b300a4 in term 1.
I20240503 08:04:38.464915 14014 consensus_queue.cc:237] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.512506 13369 catalog_manager.cc:5484] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:38.525080 13660 tablet_bootstrap.cc:654] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.517928 13861 raft_consensus.cc:482] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.536159 13665 tablet_bootstrap.cc:492] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:38.537088 13368 catalog_manager.cc:5484] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:38.544788 13748 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:38.535043 13861 raft_consensus.cc:504] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.549161 13861 leader_election.cc:288] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.549916 13618 raft_consensus.cc:3037] T 88e49c587b8a40e7a6eee4ee6566de1e P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.550740 13537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1daae5df6ae6403e895eefcf5fe05144" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
W20240503 08:04:38.552517 13434 leader_election.cc:341] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:38.537757 13665 ts_tablet_manager.cc:1367] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.191s user 0.022s sys 0.013s
I20240503 08:04:38.560597 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1daae5df6ae6403e895eefcf5fe05144" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:38.566334 13367 catalog_manager.cc:5484] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2462cc996fb441a847521d550b300a4 (127.12.6.1). New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
W20240503 08:04:38.590382 13417 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 08:04:38.594537 13665 raft_consensus.cc:348] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.599511 13665 raft_consensus.cc:374] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 08:04:38.598681 13437 leader_election.cc:341] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): Illegal state: must be running to vote when last-logged opid is not known
I20240503 08:04:38.600124 13665 raft_consensus.cc:729] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.600204 13437 leader_election.cc:302] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:38.603380 13665 consensus_queue.cc:260] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.605405 14017 raft_consensus.cc:2726] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:38.611393 13665 ts_tablet_manager.cc:1398] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.056s user 0.007s sys 0.000s
I20240503 08:04:38.617247 13661 tablet_bootstrap.cc:492] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:38.617789 13661 ts_tablet_manager.cc:1367] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.235s user 0.022s sys 0.002s
I20240503 08:04:38.620893 13661 raft_consensus.cc:348] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.627404 13661 raft_consensus.cc:374] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.628013 13661 raft_consensus.cc:729] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.629137 13661 consensus_queue.cc:260] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.642249 13665 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Bootstrap starting.
I20240503 08:04:38.644774 13661 ts_tablet_manager.cc:1398] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20240503 08:04:38.648361 13661 tablet_bootstrap.cc:492] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:38.663373 13661 tablet_bootstrap.cc:654] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.678243 13660 tablet_bootstrap.cc:492] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:38.678803 13660 ts_tablet_manager.cc:1367] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.242s user 0.012s sys 0.016s
I20240503 08:04:38.681512 13660 raft_consensus.cc:348] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.684115 13660 raft_consensus.cc:374] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.684561 13660 raft_consensus.cc:729] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.685503 13660 consensus_queue.cc:260] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.678123 13665 tablet_bootstrap.cc:654] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.691570 13660 ts_tablet_manager.cc:1398] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.012s user 0.006s sys 0.002s
I20240503 08:04:38.692807 13660 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: Bootstrap starting.
I20240503 08:04:38.694396 13661 tablet_bootstrap.cc:492] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:38.695006 13661 ts_tablet_manager.cc:1367] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.047s user 0.010s sys 0.004s
I20240503 08:04:38.697752 13661 raft_consensus.cc:348] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.698478 13661 raft_consensus.cc:374] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.698837 13661 raft_consensus.cc:729] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.699667 13661 consensus_queue.cc:260] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.704310 13660 tablet_bootstrap.cc:654] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.707690 13661 ts_tablet_manager.cc:1398] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240503 08:04:38.712482 13661 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: Bootstrap starting.
I20240503 08:04:38.718406 13661 tablet_bootstrap.cc:654] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: Neither blocks nor log segments found. Creating new log.
I20240503 08:04:38.731096 13665 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: No bootstrap required, opened a new log
I20240503 08:04:38.731694 13665 ts_tablet_manager.cc:1367] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent bootstrapping tablet: real 0.090s user 0.016s sys 0.000s
I20240503 08:04:38.737140 13660 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: No bootstrap required, opened a new log
I20240503 08:04:38.741981 13660 ts_tablet_manager.cc:1367] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: Time spent bootstrapping tablet: real 0.049s user 0.012s sys 0.019s
I20240503 08:04:38.739179 13661 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: No bootstrap required, opened a new log
I20240503 08:04:38.743574 13661 ts_tablet_manager.cc:1367] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.007s
I20240503 08:04:38.744148 13660 raft_consensus.cc:348] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.744879 13660 raft_consensus.cc:374] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.745143 13660 raft_consensus.cc:729] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.745782 13660 consensus_queue.cc:260] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.747947 13660 ts_tablet_manager.cc:1398] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20240503 08:04:38.747785 13661 raft_consensus.cc:348] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.749320 13661 raft_consensus.cc:374] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.749545 13661 raft_consensus.cc:729] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2462cc996fb441a847521d550b300a4, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.749672 13665 raft_consensus.cc:348] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.750000 13661 consensus_queue.cc:260] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.750371 13665 raft_consensus.cc:374] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 08:04:38.750671 13665 raft_consensus.cc:729] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Initialized, Role: FOLLOWER
I20240503 08:04:38.751289 13665 consensus_queue.cc:260] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.755156 14049 raft_consensus.cc:482] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.755904 14049 raft_consensus.cc:504] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.756817 13661 ts_tablet_manager.cc:1398] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240503 08:04:38.757835 14049 leader_election.cc:288] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.761237 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1daae5df6ae6403e895eefcf5fe05144" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:38.762012 13465 raft_consensus.cc:2454] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:38.763463 13511 leader_election.cc:302] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:38.764027 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1daae5df6ae6403e895eefcf5fe05144" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:38.764428 13665 ts_tablet_manager.cc:1398] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20240503 08:04:38.764348 14049 raft_consensus.cc:2781] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:38.765041 14049 raft_consensus.cc:482] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:38.765224 13619 raft_consensus.cc:2454] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 0.
I20240503 08:04:38.765432 14049 raft_consensus.cc:3037] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.771834 13999 raft_consensus.cc:482] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.772332 13999 raft_consensus.cc:504] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.772918 14049 raft_consensus.cc:504] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.774317 13999 leader_election.cc:288] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.775637 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1daae5df6ae6403e895eefcf5fe05144" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:38.776329 13465 raft_consensus.cc:3037] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.776484 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1daae5df6ae6403e895eefcf5fe05144" candidate_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5"
I20240503 08:04:38.777046 13619 raft_consensus.cc:3037] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.778810 14049 leader_election.cc:288] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.780334 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13577f32ef6f412fb64c5ae26ec2f13b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:38.781025 13467 raft_consensus.cc:2454] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:38.781176 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13577f32ef6f412fb64c5ae26ec2f13b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:38.781740 13543 raft_consensus.cc:2454] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:38.783152 13583 leader_election.cc:302] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:38.784461 13999 raft_consensus.cc:2781] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:38.784857 13999 raft_consensus.cc:482] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:38.785234 13999 raft_consensus.cc:3037] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.785971 13619 raft_consensus.cc:2454] T 1daae5df6ae6403e895eefcf5fe05144 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:38.786914 13512 leader_election.cc:302] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:38.787623 14049 raft_consensus.cc:2781] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:38.788031 13465 raft_consensus.cc:2454] T 1daae5df6ae6403e895eefcf5fe05144 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in term 1.
I20240503 08:04:38.788465 14049 raft_consensus.cc:686] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Becoming Leader. State: Replica: ba2b6f8a3dac44a0ba159bafd5cc9bc7, State: Running, Role: LEADER
I20240503 08:04:38.789168 14014 raft_consensus.cc:482] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.789654 14014 raft_consensus.cc:504] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.791800 14014 leader_election.cc:288] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917), cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:38.793620 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556938a7848d410b8ee50c324d57228f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:38.794214 13619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556938a7848d410b8ee50c324d57228f" candidate_uuid: "f2462cc996fb441a847521d550b300a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf70d9fd04894ea3812767cdb12156c5" is_pre_election: true
I20240503 08:04:38.794974 13619 raft_consensus.cc:2379] T 556938a7848d410b8ee50c324d57228f P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2462cc996fb441a847521d550b300a4 in current term 1: Already voted for candidate ba2b6f8a3dac44a0ba159bafd5cc9bc7 in this term.
I20240503 08:04:38.789475 14049 consensus_queue.cc:237] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.796020 13437 leader_election.cc:302] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2462cc996fb441a847521d550b300a4; no voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5
I20240503 08:04:38.797075 13689 raft_consensus.cc:3037] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.802943 13689 raft_consensus.cc:2726] T 556938a7848d410b8ee50c324d57228f P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 08:04:38.792835 13999 raft_consensus.cc:504] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.809027 13999 leader_election.cc:288] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.816882 13465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13577f32ef6f412fb64c5ae26ec2f13b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:38.817593 13465 raft_consensus.cc:3037] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.820040 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13577f32ef6f412fb64c5ae26ec2f13b" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:38.820701 13538 raft_consensus.cc:3037] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.826745 13368 catalog_manager.cc:5484] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2). New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:38.828346 13538 raft_consensus.cc:2454] T 13577f32ef6f412fb64c5ae26ec2f13b P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:38.830711 13583 leader_election.cc:302] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:38.831948 13465 raft_consensus.cc:2454] T 13577f32ef6f412fb64c5ae26ec2f13b P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:38.831594 13999 raft_consensus.cc:2781] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:38.833061 13999 raft_consensus.cc:686] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:38.834162 13999 consensus_queue.cc:237] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.857827 13368 catalog_manager.cc:5484] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:38.909132 14052 consensus_queue.cc:1035] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:38.911059 13761 raft_consensus.cc:482] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 08:04:38.911607 13761 raft_consensus.cc:504] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.914108 13761 leader_election.cc:288] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.914036 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c70e75f946416cb5c23f68cfe59b98" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4" is_pre_election: true
I20240503 08:04:38.915172 13467 raft_consensus.cc:2454] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:38.916193 14046 consensus_queue.cc:1035] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:38.916823 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c70e75f946416cb5c23f68cfe59b98" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" is_pre_election: true
I20240503 08:04:38.917546 13538 raft_consensus.cc:2454] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 0.
I20240503 08:04:38.918524 13583 leader_election.cc:302] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2b6f8a3dac44a0ba159bafd5cc9bc7, cf70d9fd04894ea3812767cdb12156c5; no voters:
I20240503 08:04:38.921497 14046 raft_consensus.cc:2781] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 08:04:38.921918 14046 raft_consensus.cc:482] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 08:04:38.922308 14046 raft_consensus.cc:3037] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.928320 14046 raft_consensus.cc:504] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.940001 14046 leader_election.cc:288] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Requested vote from peers f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427), ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:38.940877 13689 consensus_queue.cc:1035] T 41e689a2dc7941258f45f121e31c75bd P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:38.941798 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c70e75f946416cb5c23f68cfe59b98" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7"
I20240503 08:04:38.945021 13538 raft_consensus.cc:3037] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.942165 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c70e75f946416cb5c23f68cfe59b98" candidate_uuid: "cf70d9fd04894ea3812767cdb12156c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2462cc996fb441a847521d550b300a4"
I20240503 08:04:38.950145 13467 raft_consensus.cc:3037] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 08:04:38.955328 14044 consensus_queue.cc:1035] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:38.958679 13675 consensus_queue.cc:1035] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:38.959793 13467 raft_consensus.cc:2454] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:38.965171 13585 leader_election.cc:302] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf70d9fd04894ea3812767cdb12156c5, f2462cc996fb441a847521d550b300a4; no voters:
I20240503 08:04:38.966624 13850 raft_consensus.cc:2781] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 08:04:38.967156 13850 raft_consensus.cc:686] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Becoming Leader. State: Replica: cf70d9fd04894ea3812767cdb12156c5, State: Running, Role: LEADER
I20240503 08:04:38.968032 13850 consensus_queue.cc:237] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } }
I20240503 08:04:38.970742 13538 raft_consensus.cc:2454] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf70d9fd04894ea3812767cdb12156c5 in term 1.
I20240503 08:04:38.987670 13675 consensus_queue.cc:1035] T 556938a7848d410b8ee50c324d57228f P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:39.049590 13369 catalog_manager.cc:5484] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3). New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } }
I20240503 08:04:39.228574 13850 consensus_queue.cc:1035] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:39.240904 14046 consensus_queue.cc:1035] T 13577f32ef6f412fb64c5ae26ec2f13b P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:39.323369 14049 consensus_queue.cc:1035] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:39.336447 14049 consensus_queue.cc:1035] T 1daae5df6ae6403e895eefcf5fe05144 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:39.341811 12312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 08:04:39.398674 14107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:39.399596 14108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 08:04:39.400563 14110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 08:04:39.401491 12312 server_base.cc:1034] running on GCE node
I20240503 08:04:39.404953 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 08:04:39.405326 12312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 08:04:39.405557 12312 hybrid_clock.cc:648] HybridClock initialized: now 1714723479405539 us; error 0 us; skew 500 ppm
I20240503 08:04:39.406430 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 08:04:39.417104 12312 webserver.cc:460] Webserver started at http://127.12.6.4:43895/ using document root <none> and password file <none>
I20240503 08:04:39.417774 12312 fs_manager.cc:359] Metadata directory not provided
I20240503 08:04:39.418025 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 08:04:39.418349 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 08:04:39.419996 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-3-root/instance:
uuid: "59a35ca031314d79a9b8676ac0a0fdde"
format_stamp: "Formatted at 2024-05-03 08:04:39 on dist-test-slave-6qcz"
I20240503 08:04:39.424844 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240503 08:04:39.428531 14115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:39.429366 12312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20240503 08:04:39.429700 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-3-root
uuid: "59a35ca031314d79a9b8676ac0a0fdde"
format_stamp: "Formatted at 2024-05-03 08:04:39 on dist-test-slave-6qcz"
I20240503 08:04:39.430167 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 08:04:39.454697 12312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 08:04:39.456686 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 08:04:39.458798 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 08:04:39.462074 12312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 08:04:39.462342 12312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:39.462636 12312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 08:04:39.462865 12312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 08:04:39.491086 14044 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 08:04:39.531119 13850 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 08:04:39.625860 12312 rpc_server.cc:307] RPC server started. Bound to: 127.12.6.4:43125
I20240503 08:04:39.625401 14179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.6.4:43125 every 8 connection(s)
I20240503 08:04:39.658994 12312 auto_leader_rebalancer-test.cc:405] add a tserver: 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:39.666383 14180 heartbeater.cc:347] Connected to a master server at 127.12.6.62:35173
I20240503 08:04:39.666896 14180 heartbeater.cc:464] Registering TS with master...
I20240503 08:04:39.667840 14180 heartbeater.cc:511] Master 127.12.6.62:35173 requested a full tablet report, sending...
I20240503 08:04:39.670097 13369 ts_manager.cc:194] Registered new tserver with Master: 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125)
I20240503 08:04:39.671921 13369 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32968
I20240503 08:04:40.674876 14180 heartbeater.cc:503] Master 127.12.6.62:35173 was elected leader, sending a full tablet report...
I20240503 08:04:40.754170 13467 consensus_queue.cc:237] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:40.762300 13619 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.764053 14052 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.778398 13543 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.786000 14187 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:40.797457 14052 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.803721 13614 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.826782 13369 catalog_manager.cc:5484] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.838610 13467 consensus_queue.cc:237] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
W20240503 08:04:40.842777 13435 consensus_peers.cc:485] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f190cd50f6c2434ebf6d8e0d490eafe7. This is attempt 1: this message will repeat every 5th retry.
W20240503 08:04:40.857120 13435 consensus_peers.cc:485] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef1e609185a44d14a5473954db083b1a. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:40.857566 13619 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.858546 13545 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.858263 13538 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.859200 14052 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.860090 14192 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.871742 14192 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.877530 13545 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.887372 13619 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.906826 13368 catalog_manager.cc:5484] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:40.916064 13619 consensus_queue.cc:237] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:40.926689 13467 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.927520 13545 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.928380 13761 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.929749 14044 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.938995 14044 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
W20240503 08:04:40.947377 13584 consensus_peers.cc:485] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0786fc1af8ae401ebb5190aa117ad461. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:40.949112 13467 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.957034 13619 consensus_queue.cc:237] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:40.959956 13545 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.960211 13368 catalog_manager.cc:5484] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:40.975426 13538 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:40.976887 13467 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 08:04:40.977540 13584 consensus_peers.cc:485] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8c70e75f946416cb5c23f68cfe59b98. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:40.978912 13761 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.982934 13761 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:40.988915 14044 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.990658 13537 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:40.995213 13467 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:41.008173 13368 catalog_manager.cc:5484] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:41.155174 14211 ts_tablet_manager.cc:902] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:41.162468 14211 tablet_copy_client.cc:308] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:41.174621 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet f190cd50f6c2434ebf6d8e0d490eafe7 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:41.175356 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet f190cd50f6c2434ebf6d8e0d490eafe7 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-f190cd50f6c2434ebf6d8e0d490eafe7
I20240503 08:04:41.182602 13481 tablet_copy_source_session.cc:205] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:41.186403 14211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f190cd50f6c2434ebf6d8e0d490eafe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:41.201750 14211 tablet_copy_client.cc:783] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:41.202360 14211 tablet_copy_client.cc:647] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:41.210780 14211 tablet_copy_client.cc:523] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:41.219916 14211 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:41.243053 14211 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:41.244020 14211 tablet_bootstrap.cc:492] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:41.244689 14211 ts_tablet_manager.cc:1367] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.004s
I20240503 08:04:41.247316 14211 raft_consensus.cc:348] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.248306 14211 raft_consensus.cc:729] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:41.249017 14211 consensus_queue.cc:260] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.255554 14211 ts_tablet_manager.cc:1398] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20240503 08:04:41.257265 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-f190cd50f6c2434ebf6d8e0d490eafe7 received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:41.257650 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-f190cd50f6c2434ebf6d8e0d490eafe7 on tablet f190cd50f6c2434ebf6d8e0d490eafe7 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:41.290306 14211 ts_tablet_manager.cc:902] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:41.295068 14211 tablet_copy_client.cc:308] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:41.296499 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet ef1e609185a44d14a5473954db083b1a from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:41.297055 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet ef1e609185a44d14a5473954db083b1a from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-ef1e609185a44d14a5473954db083b1a
I20240503 08:04:41.303598 13481 tablet_copy_source_session.cc:205] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:41.312592 14211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e609185a44d14a5473954db083b1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:41.326162 14211 tablet_copy_client.cc:783] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:41.326929 14211 tablet_copy_client.cc:647] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:41.331692 14211 tablet_copy_client.cc:523] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:41.342458 14211 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:41.361167 14211 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:41.361999 14211 tablet_bootstrap.cc:492] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:41.362655 14211 ts_tablet_manager.cc:1367] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.009s
I20240503 08:04:41.365458 14211 raft_consensus.cc:348] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.366267 14211 raft_consensus.cc:729] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:41.367008 14211 consensus_queue.cc:260] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.369096 14211 ts_tablet_manager.cc:1398] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240503 08:04:41.370447 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-ef1e609185a44d14a5473954db083b1a received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:41.370857 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-ef1e609185a44d14a5473954db083b1a on tablet ef1e609185a44d14a5473954db083b1a with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:41.432874 14211 ts_tablet_manager.cc:902] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:41.443311 14211 tablet_copy_client.cc:308] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.3:33827
I20240503 08:04:41.454042 14229 ts_tablet_manager.cc:902] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:41.460423 13630 tablet_copy_service.cc:140] P cf70d9fd04894ea3812767cdb12156c5: Received BeginTabletCopySession request for tablet b8c70e75f946416cb5c23f68cfe59b98 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:36566)
I20240503 08:04:41.461086 13630 tablet_copy_service.cc:161] P cf70d9fd04894ea3812767cdb12156c5: Beginning new tablet copy session on tablet b8c70e75f946416cb5c23f68cfe59b98 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:36566: session id = 59a35ca031314d79a9b8676ac0a0fdde-b8c70e75f946416cb5c23f68cfe59b98
I20240503 08:04:41.473359 13630 tablet_copy_source_session.cc:205] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:41.476444 14211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c70e75f946416cb5c23f68cfe59b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:41.485141 14229 tablet_copy_client.cc:308] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.3:33827
I20240503 08:04:41.486642 13630 tablet_copy_service.cc:140] P cf70d9fd04894ea3812767cdb12156c5: Received BeginTabletCopySession request for tablet 0786fc1af8ae401ebb5190aa117ad461 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:36566)
I20240503 08:04:41.487213 13630 tablet_copy_service.cc:161] P cf70d9fd04894ea3812767cdb12156c5: Beginning new tablet copy session on tablet 0786fc1af8ae401ebb5190aa117ad461 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:36566: session id = 59a35ca031314d79a9b8676ac0a0fdde-0786fc1af8ae401ebb5190aa117ad461
I20240503 08:04:41.492950 14211 tablet_copy_client.cc:783] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:41.493618 14211 tablet_copy_client.cc:647] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:41.494318 13630 tablet_copy_source_session.cc:205] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:41.497944 14229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0786fc1af8ae401ebb5190aa117ad461. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:41.503224 14211 tablet_copy_client.cc:523] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:41.514528 14229 tablet_copy_client.cc:783] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:41.516413 14211 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:41.517220 14229 tablet_copy_client.cc:647] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:41.521456 14229 tablet_copy_client.cc:523] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:41.530344 14229 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:41.542317 14211 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:41.543360 14211 tablet_bootstrap.cc:492] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:41.544063 14211 ts_tablet_manager.cc:1367] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.028s user 0.020s sys 0.004s
I20240503 08:04:41.547490 14211 raft_consensus.cc:348] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.548776 14211 raft_consensus.cc:729] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:41.549654 14211 consensus_queue.cc:260] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.552520 14211 ts_tablet_manager.cc:1398] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240503 08:04:41.555362 13630 tablet_copy_service.cc:342] P cf70d9fd04894ea3812767cdb12156c5: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-b8c70e75f946416cb5c23f68cfe59b98 received from {username='slave'} at 127.0.0.1:36566
I20240503 08:04:41.555799 13630 tablet_copy_service.cc:433] P cf70d9fd04894ea3812767cdb12156c5: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-b8c70e75f946416cb5c23f68cfe59b98 on tablet b8c70e75f946416cb5c23f68cfe59b98 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:41.598640 14229 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:41.599642 14229 tablet_bootstrap.cc:492] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:41.600329 14229 ts_tablet_manager.cc:1367] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.070s user 0.016s sys 0.013s
I20240503 08:04:41.600396 14155 raft_consensus.cc:1203] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:41.603294 14229 raft_consensus.cc:348] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.604136 14229 raft_consensus.cc:729] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:41.604728 14229 consensus_queue.cc:260] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:41.615211 14229 ts_tablet_manager.cc:1398] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20240503 08:04:41.616976 13630 tablet_copy_service.cc:342] P cf70d9fd04894ea3812767cdb12156c5: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-0786fc1af8ae401ebb5190aa117ad461 received from {username='slave'} at 127.0.0.1:36566
I20240503 08:04:41.617463 13630 tablet_copy_service.cc:433] P cf70d9fd04894ea3812767cdb12156c5: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-0786fc1af8ae401ebb5190aa117ad461 on tablet 0786fc1af8ae401ebb5190aa117ad461 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:41.714367 14155 raft_consensus.cc:1203] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:41.889981 14155 raft_consensus.cc:1203] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:41.925451 14155 raft_consensus.cc:1203] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:42.000762 14192 raft_consensus.cc:1050] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:42.003484 14192 consensus_queue.cc:237] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.010133 13614 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.011186 14155 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.011799 13538 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.012018 14192 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.013296 14195 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.014710 14234 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.041545 14234 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.042901 13614 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.050976 13545 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.055255 14155 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.075534 13368 catalog_manager.cc:5484] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.139658 13467 consensus_queue.cc:237] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.149323 13614 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.150902 14155 raft_consensus.cc:1261] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.150794 14195 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:42.152853 14195 consensus_queue.cc:1035] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:42.161573 14243 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.166400 14155 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.182554 13619 raft_consensus.cc:2932] T f190cd50f6c2434ebf6d8e0d490eafe7 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.188830 13366 catalog_manager.cc:5484] T f190cd50f6c2434ebf6d8e0d490eafe7 P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.193320 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f190cd50f6c2434ebf6d8e0d490eafe7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:42.212071 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet f190cd50f6c2434ebf6d8e0d490eafe7 with delete_type TABLET_DATA_TOMBSTONED (TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52622
I20240503 08:04:42.217355 14248 tablet_replica.cc:330] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: stopping tablet replica
I20240503 08:04:42.218386 14248 raft_consensus.cc:2229] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:42.221485 14248 raft_consensus.cc:2258] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:42.225808 14248 ts_tablet_manager.cc:1875] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:42.240280 14248 ts_tablet_manager.cc:1888] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:42.240622 14248 log.cc:1196] T f190cd50f6c2434ebf6d8e0d490eafe7 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root/wals/f190cd50f6c2434ebf6d8e0d490eafe7
I20240503 08:04:42.242188 13353 catalog_manager.cc:4830] TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): tablet f190cd50f6c2434ebf6d8e0d490eafe7 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:42.313238 14242 raft_consensus.cc:1050] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:42.316939 14242 consensus_queue.cc:237] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.322455 14044 raft_consensus.cc:1050] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:42.330053 14044 consensus_queue.cc:237] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.338984 14155 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.341099 13619 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.340934 14243 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.346113 13545 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.346884 14242 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:42.349952 14234 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:42.354140 14155 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.357137 13761 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.358106 13545 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.361074 13761 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.364131 13467 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.365612 14190 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.375414 14195 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.386898 13761 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.390137 13537 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.386597 13619 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.393978 13369 catalog_manager.cc:5484] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:42.396885 13471 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.404909 13537 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.417923 14153 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.443298 14155 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.451961 13367 catalog_manager.cc:5484] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.477741 14044 raft_consensus.cc:1050] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:42.481340 14044 consensus_queue.cc:237] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.493428 13537 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.494812 13467 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.493428 14155 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:42.498425 14190 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.499362 14044 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:42.501513 14252 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20240503 08:04:42.515583 13471 consensus_queue.cc:237] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.517206 13618 consensus_queue.cc:237] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.522852 14155 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.526237 14152 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.526944 14243 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:04:42.529489 13540 raft_consensus.cc:1261] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.530144 13545 raft_consensus.cc:1261] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.528777 13761 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20240503 08:04:42.535594 14242 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.537992 14044 consensus_queue.cc:1035] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:42.539541 14195 consensus_queue.cc:1035] T ef1e609185a44d14a5473954db083b1a P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:42.552728 13545 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.557945 14190 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.559335 13761 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.562223 14155 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.565430 13537 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.558782 14152 raft_consensus.cc:2932] T ef1e609185a44d14a5473954db083b1a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.580516 14151 raft_consensus.cc:2932] T 0786fc1af8ae401ebb5190aa117ad461 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.603428 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ef1e609185a44d14a5473954db083b1a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:42.608386 13356 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0786fc1af8ae401ebb5190aa117ad461 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:42.611797 13369 catalog_manager.cc:5484] T ef1e609185a44d14a5473954db083b1a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.614147 13368 catalog_manager.cc:5484] T 0786fc1af8ae401ebb5190aa117ad461 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 08:04:42.619853 13545 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.637037 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet 0786fc1af8ae401ebb5190aa117ad461 with delete_type TABLET_DATA_TOMBSTONED (TS f2462cc996fb441a847521d550b300a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43764
I20240503 08:04:42.649087 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet ef1e609185a44d14a5473954db083b1a with delete_type TABLET_DATA_TOMBSTONED (TS cf70d9fd04894ea3812767cdb12156c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49640
I20240503 08:04:42.665309 14265 tablet_replica.cc:330] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: stopping tablet replica
I20240503 08:04:42.667320 13368 catalog_manager.cc:5484] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 08:04:42.675642 14264 tablet_replica.cc:330] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: stopping tablet replica
I20240503 08:04:42.676753 14264 raft_consensus.cc:2229] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:42.677424 14264 raft_consensus.cc:2258] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:42.689541 14264 ts_tablet_manager.cc:1875] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:42.692853 13471 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.705788 13618 consensus_queue.cc:237] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:42.709297 14264 ts_tablet_manager.cc:1888] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:42.709794 14264 log.cc:1196] T 0786fc1af8ae401ebb5190aa117ad461 P f2462cc996fb441a847521d550b300a4: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root/wals/0786fc1af8ae401ebb5190aa117ad461
I20240503 08:04:42.712182 14152 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.712747 13545 raft_consensus.cc:1261] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:42.714004 14252 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:42.716576 14190 consensus_queue.cc:1035] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:42.717053 14265 raft_consensus.cc:2229] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:42.718271 14265 raft_consensus.cc:2258] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:42.721838 14265 ts_tablet_manager.cc:1875] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:42.731837 13355 catalog_manager.cc:4830] TS f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): tablet 0786fc1af8ae401ebb5190aa117ad461 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:42.740451 14265 ts_tablet_manager.cc:1888] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:42.741385 14265 log.cc:1196] T ef1e609185a44d14a5473954db083b1a P cf70d9fd04894ea3812767cdb12156c5: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root/wals/ef1e609185a44d14a5473954db083b1a
I20240503 08:04:42.740525 14252 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.740938 13537 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.743417 13356 catalog_manager.cc:4830] TS cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): tablet ef1e609185a44d14a5473954db083b1a (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:42.758915 13356 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b8c70e75f946416cb5c23f68cfe59b98 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:42.759925 14152 raft_consensus.cc:2932] T b8c70e75f946416cb5c23f68cfe59b98 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.776584 13368 catalog_manager.cc:5484] T b8c70e75f946416cb5c23f68cfe59b98 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:42.785761 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet b8c70e75f946416cb5c23f68cfe59b98 with delete_type TABLET_DATA_TOMBSTONED (TS f2462cc996fb441a847521d550b300a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43764
I20240503 08:04:42.786775 14264 tablet_replica.cc:330] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: stopping tablet replica
I20240503 08:04:42.787755 14264 raft_consensus.cc:2229] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:42.788326 14264 raft_consensus.cc:2258] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:42.790828 14264 ts_tablet_manager.cc:1875] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:42.802843 14264 ts_tablet_manager.cc:1888] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:42.803190 14264 log.cc:1196] T b8c70e75f946416cb5c23f68cfe59b98 P f2462cc996fb441a847521d550b300a4: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root/wals/b8c70e75f946416cb5c23f68cfe59b98
I20240503 08:04:42.804415 13355 catalog_manager.cc:4830] TS f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): tablet b8c70e75f946416cb5c23f68cfe59b98 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:43.836301 13467 consensus_queue.cc:237] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
W20240503 08:04:43.843057 13435 consensus_peers.cc:485] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 008315f470c24029bd9271aad3c67a09. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:43.844099 13618 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:43.844100 13540 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:43.846344 14192 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:43.848181 14234 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:43.854588 14243 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.856281 13618 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.860630 13545 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.868038 13467 consensus_queue.cc:237] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:43.875455 13614 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 08:04:43.875455 13435 consensus_peers.cc:485] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8497edff1aaf45e48da16ad73a649c6d. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:43.876909 14268 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:43.889791 13369 catalog_manager.cc:5484] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:43.900317 13545 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:43.904441 14192 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:43.928575 14234 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.937494 13545 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.943373 13614 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.954321 13467 consensus_queue.cc:237] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:43.956777 13366 catalog_manager.cc:5484] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
W20240503 08:04:43.974306 13435 consensus_peers.cc:485] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c5d2ce341b749f8b2fd0c4ac23b4018. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:43.974395 13618 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:43.975912 14243 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:43.977363 13545 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:43.980322 14243 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:43.988075 14268 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.989315 13618 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:43.994856 13545 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:44.001521 13614 consensus_queue.cc:237] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.012156 13369 catalog_manager.cc:5484] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
W20240503 08:04:44.029347 13584 consensus_peers.cc:485] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 768f7ed6c5cc40bd947c30cf5450c8f1. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:44.029582 13467 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:44.030231 13545 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:44.034315 14252 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:44.036075 14252 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:44.048202 14266 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:44.049825 13537 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:44.051244 13467 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:44.069237 13369 catalog_manager.cc:5484] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:44.272881 14285 ts_tablet_manager.cc:902] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:44.283303 14285 tablet_copy_client.cc:308] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:44.285038 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet 008315f470c24029bd9271aad3c67a09 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:44.285575 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet 008315f470c24029bd9271aad3c67a09 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-008315f470c24029bd9271aad3c67a09
I20240503 08:04:44.293279 13481 tablet_copy_source_session.cc:205] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:44.297945 14285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 008315f470c24029bd9271aad3c67a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:44.307885 14290 ts_tablet_manager.cc:902] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:44.322609 14285 tablet_copy_client.cc:783] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:44.322849 14290 tablet_copy_client.cc:308] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:44.323266 14285 tablet_copy_client.cc:647] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:44.324373 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet 8497edff1aaf45e48da16ad73a649c6d from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:44.324936 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet 8497edff1aaf45e48da16ad73a649c6d from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-8497edff1aaf45e48da16ad73a649c6d
I20240503 08:04:44.337596 13481 tablet_copy_source_session.cc:205] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:44.340970 14290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8497edff1aaf45e48da16ad73a649c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:44.338907 14285 tablet_copy_client.cc:523] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:44.354393 14285 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:44.356422 14290 tablet_copy_client.cc:783] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:44.357307 14290 tablet_copy_client.cc:647] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:44.366017 14290 tablet_copy_client.cc:523] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:44.371791 14295 ts_tablet_manager.cc:902] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:44.375241 14290 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:44.389493 14285 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:44.390951 14285 tablet_bootstrap.cc:492] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:44.391912 14285 ts_tablet_manager.cc:1367] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.038s user 0.023s sys 0.009s
I20240503 08:04:44.399981 14285 raft_consensus.cc:348] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.400895 14285 raft_consensus.cc:729] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:44.401520 14285 consensus_queue.cc:260] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.405725 14295 tablet_copy_client.cc:308] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:44.412290 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:44.412863 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-4c5d2ce341b749f8b2fd0c4ac23b4018
I20240503 08:04:44.418534 13481 tablet_copy_source_session.cc:205] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:44.412832 14290 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:44.421133 14290 tablet_bootstrap.cc:492] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:44.421489 14295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:44.421877 14290 ts_tablet_manager.cc:1367] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.047s user 0.023s sys 0.005s
I20240503 08:04:44.424743 14290 raft_consensus.cc:348] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.425648 14290 raft_consensus.cc:729] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:44.426296 14290 consensus_queue.cc:260] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.430292 14285 ts_tablet_manager.cc:1398] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.038s user 0.009s sys 0.021s
I20240503 08:04:44.431480 14290 ts_tablet_manager.cc:1398] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.009s user 0.005s sys 0.004s
I20240503 08:04:44.432559 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-008315f470c24029bd9271aad3c67a09 received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:44.433013 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-008315f470c24029bd9271aad3c67a09 on tablet 008315f470c24029bd9271aad3c67a09 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:44.436048 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-8497edff1aaf45e48da16ad73a649c6d received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:44.436472 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-8497edff1aaf45e48da16ad73a649c6d on tablet 8497edff1aaf45e48da16ad73a649c6d with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:44.444447 14295 tablet_copy_client.cc:783] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:44.445057 14295 tablet_copy_client.cc:647] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:44.451156 14295 tablet_copy_client.cc:523] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:44.459605 14295 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:44.479043 14295 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:44.479916 14295 tablet_bootstrap.cc:492] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:44.480535 14295 ts_tablet_manager.cc:1367] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.003s
I20240503 08:04:44.483160 14295 raft_consensus.cc:348] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.484023 14295 raft_consensus.cc:729] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:44.484665 14295 consensus_queue.cc:260] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.486505 14295 ts_tablet_manager.cc:1398] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:44.488128 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-4c5d2ce341b749f8b2fd0c4ac23b4018 received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:44.488557 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-4c5d2ce341b749f8b2fd0c4ac23b4018 on tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:44.495033 14295 ts_tablet_manager.cc:902] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
I20240503 08:04:44.503279 14295 tablet_copy_client.cc:308] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.3:33827
I20240503 08:04:44.506399 13630 tablet_copy_service.cc:140] P cf70d9fd04894ea3812767cdb12156c5: Received BeginTabletCopySession request for tablet 768f7ed6c5cc40bd947c30cf5450c8f1 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:36566)
I20240503 08:04:44.507428 13630 tablet_copy_service.cc:161] P cf70d9fd04894ea3812767cdb12156c5: Beginning new tablet copy session on tablet 768f7ed6c5cc40bd947c30cf5450c8f1 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:36566: session id = 59a35ca031314d79a9b8676ac0a0fdde-768f7ed6c5cc40bd947c30cf5450c8f1
I20240503 08:04:44.514118 13630 tablet_copy_source_session.cc:205] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:44.519732 14295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768f7ed6c5cc40bd947c30cf5450c8f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:44.535989 14295 tablet_copy_client.cc:783] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:44.536540 14295 tablet_copy_client.cc:647] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:44.540464 14295 tablet_copy_client.cc:523] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:44.550489 14295 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:44.571161 14295 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:44.572104 14295 tablet_bootstrap.cc:492] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:44.572702 14295 ts_tablet_manager.cc:1367] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.022s user 0.022s sys 0.000s
I20240503 08:04:44.575477 14295 raft_consensus.cc:348] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.576313 14295 raft_consensus.cc:729] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:44.576942 14295 consensus_queue.cc:260] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:44.580004 14295 ts_tablet_manager.cc:1398] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20240503 08:04:44.581876 13630 tablet_copy_service.cc:342] P cf70d9fd04894ea3812767cdb12156c5: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-768f7ed6c5cc40bd947c30cf5450c8f1 received from {username='slave'} at 127.0.0.1:36566
I20240503 08:04:44.582278 13630 tablet_copy_service.cc:433] P cf70d9fd04894ea3812767cdb12156c5: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-768f7ed6c5cc40bd947c30cf5450c8f1 on tablet 768f7ed6c5cc40bd947c30cf5450c8f1 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:44.826750 14155 raft_consensus.cc:1203] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:44.876721 14155 raft_consensus.cc:1203] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:44.971367 14155 raft_consensus.cc:1203] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:44.990489 14155 raft_consensus.cc:1203] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:45.288400 14234 raft_consensus.cc:1050] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:45.290823 14234 consensus_queue.cc:237] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.293246 14243 raft_consensus.cc:1050] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:45.297756 14243 consensus_queue.cc:237] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.304127 14155 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.307461 14268 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:45.309772 13614 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.312886 14268 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:45.327540 14243 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.329722 13618 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.330530 13540 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.331612 14314 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.332780 14268 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:45.333173 13614 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.331318 14155 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.350991 14268 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:45.355322 13545 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.356459 14152 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.357692 14315 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.358680 13369 catalog_manager.cc:5484] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.392772 14268 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.394251 13538 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.405149 13619 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.422771 14268 raft_consensus.cc:1050] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:45.425315 14268 consensus_queue.cc:237] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.425637 13367 catalog_manager.cc:5484] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.428968 13545 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.448457 13614 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.448335 14152 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.451155 14268 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:45.453747 14315 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.458829 13540 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.460352 14315 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.468725 14155 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.490253 14234 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.494884 13545 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.502987 14310 raft_consensus.cc:1050] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:45.504945 14155 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.507752 14310 consensus_queue.cc:237] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.524075 13618 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.527823 14155 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.528719 13540 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.529742 13465 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:45.530036 14310 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.530912 14044 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.546310 14252 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:45.554343 13367 catalog_manager.cc:5484] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.590243 14310 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.596729 14152 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.598119 13467 consensus_queue.cc:237] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.603310 14151 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.604828 14234 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:45.607982 13613 raft_consensus.cc:1261] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.607738 13540 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.609833 14323 consensus_queue.cc:1035] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:45.617955 13467 consensus_queue.cc:237] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.625766 14234 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.626610 13369 catalog_manager.cc:5484] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.630726 14152 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.654615 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8497edff1aaf45e48da16ad73a649c6d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:45.664647 13471 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.666599 14152 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.670010 13619 raft_consensus.cc:1261] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.671401 14323 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:45.672909 14234 consensus_queue.cc:1035] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:45.670372 13367 catalog_manager.cc:5484] T 8497edff1aaf45e48da16ad73a649c6d P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 08:04:45.680671 13613 raft_consensus.cc:2932] T 8497edff1aaf45e48da16ad73a649c6d P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.681337 14313 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.679828 13614 consensus_queue.cc:237] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.690280 14151 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.690827 14155 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.695273 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:45.695081 14337 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:04:45.699651 13538 raft_consensus.cc:1261] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.706360 14252 consensus_queue.cc:1035] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:45.717507 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet 8497edff1aaf45e48da16ad73a649c6d with delete_type TABLET_DATA_TOMBSTONED (TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52622
I20240503 08:04:45.728394 14252 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.733585 14152 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.727623 13614 raft_consensus.cc:2932] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.731611 13540 raft_consensus.cc:2932] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.753783 13356 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 768f7ed6c5cc40bd947c30cf5450c8f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:45.755321 13367 catalog_manager.cc:5484] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.764981 14339 tablet_replica.cc:330] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: stopping tablet replica
I20240503 08:04:45.766079 14339 raft_consensus.cc:2229] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:45.766703 14339 raft_consensus.cc:2258] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:45.771577 14339 ts_tablet_manager.cc:1875] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:45.773478 13368 catalog_manager.cc:5484] T 768f7ed6c5cc40bd947c30cf5450c8f1 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.774261 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 with delete_type TABLET_DATA_TOMBSTONED (TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52622
I20240503 08:04:45.790854 14339 ts_tablet_manager.cc:1888] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:45.791363 14339 log.cc:1196] T 8497edff1aaf45e48da16ad73a649c6d P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root/wals/8497edff1aaf45e48da16ad73a649c6d
I20240503 08:04:45.793395 13353 catalog_manager.cc:4830] TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): tablet 8497edff1aaf45e48da16ad73a649c6d (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:45.805493 14339 tablet_replica.cc:330] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: stopping tablet replica
I20240503 08:04:45.807626 14339 raft_consensus.cc:2229] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:45.808261 14339 raft_consensus.cc:2258] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:45.813218 14339 ts_tablet_manager.cc:1875] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:45.820966 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet 768f7ed6c5cc40bd947c30cf5450c8f1 with delete_type TABLET_DATA_TOMBSTONED (TS f2462cc996fb441a847521d550b300a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43764
I20240503 08:04:45.832134 14339 ts_tablet_manager.cc:1888] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:45.832924 14339 log.cc:1196] T 4c5d2ce341b749f8b2fd0c4ac23b4018 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root/wals/4c5d2ce341b749f8b2fd0c4ac23b4018
I20240503 08:04:45.834787 13353 catalog_manager.cc:4830] TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): tablet 4c5d2ce341b749f8b2fd0c4ac23b4018 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:45.835494 14340 tablet_replica.cc:330] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: stopping tablet replica
I20240503 08:04:45.836283 14340 raft_consensus.cc:2229] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:45.836827 14340 raft_consensus.cc:2258] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:45.839555 14340 ts_tablet_manager.cc:1875] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:45.853385 14340 ts_tablet_manager.cc:1888] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:45.853786 14340 log.cc:1196] T 768f7ed6c5cc40bd947c30cf5450c8f1 P f2462cc996fb441a847521d550b300a4: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root/wals/768f7ed6c5cc40bd947c30cf5450c8f1
I20240503 08:04:45.855296 13355 catalog_manager.cc:4830] TS f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): tablet 768f7ed6c5cc40bd947c30cf5450c8f1 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:45.895576 13467 consensus_queue.cc:237] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:45.900352 13538 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.900624 14152 raft_consensus.cc:1261] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:45.902734 14315 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:45.903820 14243 consensus_queue.cc:1035] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:04:45.912281 14243 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.913955 13538 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.914225 14152 raft_consensus.cc:2932] T 008315f470c24029bd9271aad3c67a09 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:45.927739 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 008315f470c24029bd9271aad3c67a09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:45.930867 13366 catalog_manager.cc:5484] T 008315f470c24029bd9271aad3c67a09 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 08:04:45.939515 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 008315f470c24029bd9271aad3c67a09 with delete_type TABLET_DATA_TOMBSTONED (TS cf70d9fd04894ea3812767cdb12156c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49640
I20240503 08:04:45.941300 14341 tablet_replica.cc:330] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: stopping tablet replica
I20240503 08:04:45.942389 14341 raft_consensus.cc:2229] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:45.943212 14341 raft_consensus.cc:2258] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:45.946331 14341 ts_tablet_manager.cc:1875] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:45.958909 14341 ts_tablet_manager.cc:1888] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:45.959231 14341 log.cc:1196] T 008315f470c24029bd9271aad3c67a09 P cf70d9fd04894ea3812767cdb12156c5: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root/wals/008315f470c24029bd9271aad3c67a09
I20240503 08:04:45.960778 13356 catalog_manager.cc:4830] TS cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): tablet 008315f470c24029bd9271aad3c67a09 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:46.993345 13467 consensus_queue.cc:237] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:46.998718 13614 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 08:04:46.999120 13435 consensus_peers.cc:485] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d88e8a1e1ad45f9a3cdf022a28162fb. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:47.000366 14243 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:47.001190 13538 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:47.002373 14243 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:47.008953 14313 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.011090 13613 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.014385 13538 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.029000 13368 catalog_manager.cc:5484] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:47.052088 13540 consensus_queue.cc:237] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.088272 13467 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:47.095160 14338 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:47.097579 13613 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 08:04:47.101233 13510 consensus_peers.cc:485] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 161465c92bbf4897bc9944c2b9869a9a. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:47.102142 14338 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:47.109146 14352 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.112672 13467 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.126341 13471 consensus_queue.cc:237] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.128170 13367 catalog_manager.cc:5484] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:47.141559 13613 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.155427 13613 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:47.156795 14234 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20240503 08:04:47.161033 13435 consensus_peers.cc:485] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ef1acee9ffb4310bde01caa87d35754. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:47.165457 13545 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:47.167039 14243 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.169152 13613 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.174862 14361 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:47.184954 13471 consensus_queue.cc:237] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.185271 13366 catalog_manager.cc:5484] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240503 08:04:47.211831 13435 consensus_peers.cc:485] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aae93c20681b4e4fa9b521edbb9e65e0. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:47.212843 13545 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:47.212567 13538 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.216270 13613 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:47.216720 14361 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:47.218811 14361 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:47.229873 14234 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.233484 13613 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.253954 13540 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:47.254657 13368 catalog_manager.cc:5484] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:47.429216 14367 ts_tablet_manager.cc:902] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:47.437206 14367 tablet_copy_client.cc:308] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:47.438588 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:47.439025 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet 3d88e8a1e1ad45f9a3cdf022a28162fb from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-3d88e8a1e1ad45f9a3cdf022a28162fb
I20240503 08:04:47.445327 13481 tablet_copy_source_session.cc:205] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:47.448639 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:47.462824 14367 tablet_copy_client.cc:783] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:47.463426 14367 tablet_copy_client.cc:647] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:47.467152 14367 tablet_copy_client.cc:523] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:47.475723 14367 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:47.494729 14367 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:47.495630 14367 tablet_bootstrap.cc:492] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:47.496181 14367 ts_tablet_manager.cc:1367] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.011s
I20240503 08:04:47.498628 14367 raft_consensus.cc:348] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.499555 14367 raft_consensus.cc:729] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:47.500077 14367 consensus_queue.cc:260] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.502930 14367 ts_tablet_manager.cc:1398] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20240503 08:04:47.505154 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-3d88e8a1e1ad45f9a3cdf022a28162fb received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:47.505576 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-3d88e8a1e1ad45f9a3cdf022a28162fb on tablet 3d88e8a1e1ad45f9a3cdf022a28162fb with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:47.543440 14367 ts_tablet_manager.cc:902] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
I20240503 08:04:47.547037 14367 tablet_copy_client.cc:308] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.2:44917
I20240503 08:04:47.558696 13556 tablet_copy_service.cc:140] P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Received BeginTabletCopySession request for tablet 161465c92bbf4897bc9944c2b9869a9a from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:34228)
I20240503 08:04:47.559320 13556 tablet_copy_service.cc:161] P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Beginning new tablet copy session on tablet 161465c92bbf4897bc9944c2b9869a9a from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:34228: session id = 59a35ca031314d79a9b8676ac0a0fdde-161465c92bbf4897bc9944c2b9869a9a
I20240503 08:04:47.565477 13556 tablet_copy_source_session.cc:205] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:47.569188 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161465c92bbf4897bc9944c2b9869a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:47.583544 14367 tablet_copy_client.cc:783] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:47.584087 14367 tablet_copy_client.cc:647] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:47.588534 14367 tablet_copy_client.cc:523] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:47.599500 14367 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:47.617910 14367 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:47.618818 14367 tablet_bootstrap.cc:492] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:47.619513 14367 ts_tablet_manager.cc:1367] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.002s
I20240503 08:04:47.621737 14367 raft_consensus.cc:348] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.622555 14367 raft_consensus.cc:729] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:47.623070 14367 consensus_queue.cc:260] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.625346 14367 ts_tablet_manager.cc:1398] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240503 08:04:47.628408 13556 tablet_copy_service.cc:342] P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-161465c92bbf4897bc9944c2b9869a9a received from {username='slave'} at 127.0.0.1:34228
I20240503 08:04:47.628834 13556 tablet_copy_service.cc:433] P ba2b6f8a3dac44a0ba159bafd5cc9bc7: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-161465c92bbf4897bc9944c2b9869a9a on tablet 161465c92bbf4897bc9944c2b9869a9a with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:47.703326 14367 ts_tablet_manager.cc:902] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:47.706836 14367 tablet_copy_client.cc:308] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:47.710139 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet aae93c20681b4e4fa9b521edbb9e65e0 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:47.710661 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet aae93c20681b4e4fa9b521edbb9e65e0 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-aae93c20681b4e4fa9b521edbb9e65e0
I20240503 08:04:47.714330 14384 ts_tablet_manager.cc:902] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
I20240503 08:04:47.717702 13481 tablet_copy_source_session.cc:205] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:47.719204 14384 tablet_copy_client.cc:308] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Beginning tablet copy session from remote peer at address 127.12.6.1:34427
I20240503 08:04:47.721328 13481 tablet_copy_service.cc:140] P f2462cc996fb441a847521d550b300a4: Received BeginTabletCopySession request for tablet 0ef1acee9ffb4310bde01caa87d35754 from peer 59a35ca031314d79a9b8676ac0a0fdde ({username='slave'} at 127.0.0.1:41550)
I20240503 08:04:47.722916 13481 tablet_copy_service.cc:161] P f2462cc996fb441a847521d550b300a4: Beginning new tablet copy session on tablet 0ef1acee9ffb4310bde01caa87d35754 from peer 59a35ca031314d79a9b8676ac0a0fdde at {username='slave'} at 127.0.0.1:41550: session id = 59a35ca031314d79a9b8676ac0a0fdde-0ef1acee9ffb4310bde01caa87d35754
I20240503 08:04:47.723596 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aae93c20681b4e4fa9b521edbb9e65e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:47.731423 13481 tablet_copy_source_session.cc:205] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 08:04:47.734504 14384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef1acee9ffb4310bde01caa87d35754. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 08:04:47.741250 14367 tablet_copy_client.cc:783] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:47.741871 14367 tablet_copy_client.cc:647] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:47.747632 14367 tablet_copy_client.cc:523] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:47.752094 14384 tablet_copy_client.cc:783] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 0 data blocks...
I20240503 08:04:47.752722 14384 tablet_copy_client.cc:647] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Starting download of 1 WAL segments...
I20240503 08:04:47.756774 14384 tablet_copy_client.cc:523] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 08:04:47.762825 14367 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:47.773620 14384 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap starting.
I20240503 08:04:47.793059 14367 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:47.793929 14367 tablet_bootstrap.cc:492] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:47.794543 14367 ts_tablet_manager.cc:1367] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.005s
I20240503 08:04:47.795720 14384 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 08:04:47.796567 14384 tablet_bootstrap.cc:492] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: Bootstrap complete.
I20240503 08:04:47.797230 14384 ts_tablet_manager.cc:1367] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent bootstrapping tablet: real 0.024s user 0.017s sys 0.005s
I20240503 08:04:47.797155 14367 raft_consensus.cc:348] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.798139 14367 raft_consensus.cc:729] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:47.798743 14367 consensus_queue.cc:260] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.800083 14384 raft_consensus.cc:348] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.801002 14384 raft_consensus.cc:729] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59a35ca031314d79a9b8676ac0a0fdde, State: Initialized, Role: LEARNER
I20240503 08:04:47.803375 14384 consensus_queue.cc:260] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:47.806772 14367 ts_tablet_manager.cc:1398] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.012s user 0.009s sys 0.000s
I20240503 08:04:47.808084 14384 ts_tablet_manager.cc:1398] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20240503 08:04:47.808495 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-aae93c20681b4e4fa9b521edbb9e65e0 received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:47.808928 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-aae93c20681b4e4fa9b521edbb9e65e0 on tablet aae93c20681b4e4fa9b521edbb9e65e0 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:47.813921 13481 tablet_copy_service.cc:342] P f2462cc996fb441a847521d550b300a4: Request end of tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-0ef1acee9ffb4310bde01caa87d35754 received from {username='slave'} at 127.0.0.1:41550
I20240503 08:04:47.814332 13481 tablet_copy_service.cc:433] P f2462cc996fb441a847521d550b300a4: ending tablet copy session 59a35ca031314d79a9b8676ac0a0fdde-0ef1acee9ffb4310bde01caa87d35754 on tablet 0ef1acee9ffb4310bde01caa87d35754 with peer 59a35ca031314d79a9b8676ac0a0fdde
I20240503 08:04:47.836134 14152 raft_consensus.cc:1203] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:48.030229 14152 raft_consensus.cc:1203] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:48.167248 14152 raft_consensus.cc:1203] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 08:04:48.259225 14152 raft_consensus.cc:1203] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240503 08:04:48.358345 13585 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.6.1:34427, user_credentials={real_user=slave}} blocked reactor thread for 105888us
I20240503 08:04:48.412797 14394 raft_consensus.cc:1050] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:48.417562 14394 consensus_queue.cc:237] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.424278 13538 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.424278 13614 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.427305 14234 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:48.428519 14152 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.431275 14234 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:48.436218 14395 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20240503 08:04:48.464169 14396 raft_consensus.cc:1050] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:48.469650 14396 consensus_queue.cc:237] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.472795 14234 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.483333 13545 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.506060 13614 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.507769 14152 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.518684 13368 catalog_manager.cc:5484] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.538905 13614 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.540520 13471 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.540709 14356 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:48.542048 14152 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.542274 14398 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:48.543785 14396 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:48.565773 14356 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.577780 14155 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.590607 13619 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.593156 13471 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.626721 13471 consensus_queue.cc:237] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.626817 13369 catalog_manager.cc:5484] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 08:04:48.633199 14155 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.636446 14395 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:48.640513 13545 raft_consensus.cc:1261] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.642266 14243 consensus_queue.cc:1035] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:48.643595 14395 raft_consensus.cc:1050] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:48.648743 14243 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.652765 14395 consensus_queue.cc:237] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.653214 14152 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.668376 13545 raft_consensus.cc:2932] T 3d88e8a1e1ad45f9a3cdf022a28162fb P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.672938 13540 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.674147 13614 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.674540 14394 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:48.676128 14361 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:48.679729 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:48.692499 13367 catalog_manager.cc:5484] T 3d88e8a1e1ad45f9a3cdf022a28162fb P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.700791 13538 consensus_queue.cc:237] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.702510 14155 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.704761 14393 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 08:04:48.707021 14152 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.711584 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d88e8a1e1ad45f9a3cdf022a28162fb with delete_type TABLET_DATA_TOMBSTONED (TS cf70d9fd04894ea3812767cdb12156c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49640
I20240503 08:04:48.713994 14396 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:48.715685 13616 raft_consensus.cc:1261] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.716243 14243 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.719172 14399 consensus_queue.cc:1035] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:04:48.718297 13616 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.725062 14396 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.726841 14151 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.728449 13545 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.742910 13353 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 161465c92bbf4897bc9944c2b9869a9a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:48.751912 14417 tablet_replica.cc:330] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: stopping tablet replica
I20240503 08:04:48.755873 13368 catalog_manager.cc:5484] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240503 08:04:48.755647 13614 raft_consensus.cc:2932] T 161465c92bbf4897bc9944c2b9869a9a P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.759568 14417 raft_consensus.cc:2229] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:48.762660 14417 raft_consensus.cc:2258] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:48.774832 13369 catalog_manager.cc:5484] T 161465c92bbf4897bc9944c2b9869a9a P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 3 to 4, VOTER f2462cc996fb441a847521d550b300a4 (127.12.6.1) evicted. New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.779664 14361 raft_consensus.cc:1050] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4: attempting to promote NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde to VOTER
I20240503 08:04:48.782334 14361 consensus_queue.cc:237] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.784709 14151 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.792847 14152 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 LEARNER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.803279 13457 tablet_service.cc:1514] Processing DeleteTablet for tablet 161465c92bbf4897bc9944c2b9869a9a with delete_type TABLET_DATA_TOMBSTONED (TS f2462cc996fb441a847521d550b300a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43764
I20240503 08:04:48.805514 14420 tablet_replica.cc:330] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: stopping tablet replica
I20240503 08:04:48.806607 14420 raft_consensus.cc:2229] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:48.807356 14420 raft_consensus.cc:2258] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:48.793006 13545 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.809819 14361 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:48.793821 13614 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 08:04:48.799960 14417 ts_tablet_manager.cc:1875] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:48.812794 14395 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:48.815795 14361 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 08:04:48.825980 13467 consensus_queue.cc:237] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.835255 14152 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.836406 13538 raft_consensus.cc:1261] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.838002 14234 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.840981 14395 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:48.844311 13616 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.846236 14361 consensus_queue.cc:1035] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240503 08:04:48.852178 14394 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.858582 14420 ts_tablet_manager.cc:1875] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:48.864111 14151 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.867516 13540 raft_consensus.cc:2932] T 0ef1acee9ffb4310bde01caa87d35754 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.878779 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0ef1acee9ffb4310bde01caa87d35754 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:48.870885 13367 catalog_manager.cc:5484] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.881134 14152 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.902688 13367 catalog_manager.cc:5484] T 0ef1acee9ffb4310bde01caa87d35754 P 59a35ca031314d79a9b8676ac0a0fdde reported cstate change: config changed from index 3 to 4, VOTER cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.904011 13545 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.907750 14417 ts_tablet_manager.cc:1888] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:48.910179 14417 log.cc:1196] T 3d88e8a1e1ad45f9a3cdf022a28162fb P cf70d9fd04894ea3812767cdb12156c5: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root/wals/3d88e8a1e1ad45f9a3cdf022a28162fb
I20240503 08:04:48.913067 13356 catalog_manager.cc:4830] TS cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): tablet 3d88e8a1e1ad45f9a3cdf022a28162fb (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:48.920049 14420 ts_tablet_manager.cc:1888] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:48.920661 14420 log.cc:1196] T 161465c92bbf4897bc9944c2b9869a9a P f2462cc996fb441a847521d550b300a4: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-0-root/wals/161465c92bbf4897bc9944c2b9869a9a
I20240503 08:04:48.923712 13355 catalog_manager.cc:4830] TS f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427): tablet 161465c92bbf4897bc9944c2b9869a9a (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:48.929358 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ef1acee9ffb4310bde01caa87d35754 with delete_type TABLET_DATA_TOMBSTONED (TS cf70d9fd04894ea3812767cdb12156c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49640
I20240503 08:04:48.930406 14417 tablet_replica.cc:330] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: stopping tablet replica
I20240503 08:04:48.931490 14417 raft_consensus.cc:2229] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:48.932119 14417 raft_consensus.cc:2258] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:48.937584 14417 ts_tablet_manager.cc:1875] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:48.944473 13467 consensus_queue.cc:237] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } }
I20240503 08:04:48.969926 14152 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.972636 14393 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:48.974041 13614 raft_consensus.cc:1261] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 08:04:48.976023 14395 consensus_queue.cc:1035] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 08:04:48.980906 14395 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.983209 14151 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P 59a35ca031314d79a9b8676ac0a0fdde [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.984540 13614 raft_consensus.cc:2932] T aae93c20681b4e4fa9b521edbb9e65e0 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } } }
I20240503 08:04:48.989257 14417 ts_tablet_manager.cc:1888] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:48.989770 14417 log.cc:1196] T 0ef1acee9ffb4310bde01caa87d35754 P cf70d9fd04894ea3812767cdb12156c5: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-2-root/wals/0ef1acee9ffb4310bde01caa87d35754
I20240503 08:04:48.992761 13356 catalog_manager.cc:4830] TS cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827): tablet 0ef1acee9ffb4310bde01caa87d35754 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:48.998508 13355 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet aae93c20681b4e4fa9b521edbb9e65e0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 08:04:49.002269 13366 catalog_manager.cc:5484] T aae93c20681b4e4fa9b521edbb9e65e0 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index 3 to 4, VOTER ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2) evicted. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 08:04:49.012653 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet aae93c20681b4e4fa9b521edbb9e65e0 with delete_type TABLET_DATA_TOMBSTONED (TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52622
I20240503 08:04:49.014783 14424 tablet_replica.cc:330] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: stopping tablet replica
I20240503 08:04:49.016145 14424 raft_consensus.cc:2229] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 08:04:49.021049 14424 raft_consensus.cc:2258] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 08:04:49.024612 14424 ts_tablet_manager.cc:1875] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 08:04:49.038219 14424 ts_tablet_manager.cc:1888] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 08:04:49.038556 14424 log.cc:1196] T aae93c20681b4e4fa9b521edbb9e65e0 P ba2b6f8a3dac44a0ba159bafd5cc9bc7: Deleting WAL directory at /tmp/dist-test-taskNRCDxF/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714723407607891-12312-0/minicluster-data/ts-1-root/wals/aae93c20681b4e4fa9b521edbb9e65e0
I20240503 08:04:49.040853 13353 catalog_manager.cc:4830] TS ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917): tablet aae93c20681b4e4fa9b521edbb9e65e0 (table test-workload [id=1898f06dfb474368bb2b92a9245afac3]) successfully deleted
I20240503 08:04:50.072819 13614 consensus_queue.cc:237] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:50.078617 13545 raft_consensus.cc:1261] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 08:04:50.078842 13584 consensus_peers.cc:485] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4999c55f4034c3d9997841353a35afd. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:50.081751 14425 consensus_queue.cc:1035] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:50.083333 13467 raft_consensus.cc:1261] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.084614 14400 consensus_queue.cc:1035] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:50.094360 14252 raft_consensus.cc:2932] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.096094 13467 raft_consensus.cc:2932] T d4999c55f4034c3d9997841353a35afd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.101372 13545 raft_consensus.cc:2932] T d4999c55f4034c3d9997841353a35afd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.113334 13471 consensus_queue.cc:237] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
I20240503 08:04:50.114176 13366 catalog_manager.cc:5484] T d4999c55f4034c3d9997841353a35afd P cf70d9fd04894ea3812767cdb12156c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240503 08:04:50.136128 13435 consensus_peers.cc:485] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 248eecaf8416442794c9558a2d73bdc9. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:50.139236 13545 raft_consensus.cc:1261] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.139688 13614 raft_consensus.cc:1261] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer f2462cc996fb441a847521d550b300a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.141108 14428 consensus_queue.cc:1035] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:50.142585 14428 consensus_queue.cc:1035] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:50.152537 14428 raft_consensus.cc:2932] T 248eecaf8416442794c9558a2d73bdc9 P f2462cc996fb441a847521d550b300a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.155907 13545 raft_consensus.cc:2932] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.156961 13614 raft_consensus.cc:2932] T 248eecaf8416442794c9558a2d73bdc9 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.171448 13368 catalog_manager.cc:5484] T 248eecaf8416442794c9558a2d73bdc9 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "f2462cc996fb441a847521d550b300a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.175480 13616 consensus_queue.cc:237] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
W20240503 08:04:50.195150 13584 consensus_peers.cc:485] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f02eda12b9544888367d6206e3b41dd. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:50.196552 13467 raft_consensus.cc:1261] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.198014 14425 consensus_queue.cc:1035] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:50.196662 13545 raft_consensus.cc:1261] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Refusing update from remote peer cf70d9fd04894ea3812767cdb12156c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.200838 14252 consensus_queue.cc:1035] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:50.211629 14252 raft_consensus.cc:2932] T 6f02eda12b9544888367d6206e3b41dd P cf70d9fd04894ea3812767cdb12156c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.213179 13538 raft_consensus.cc:2932] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.224913 13467 raft_consensus.cc:2932] T 6f02eda12b9544888367d6206e3b41dd P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.232249 13367 catalog_manager.cc:5484] T 6f02eda12b9544888367d6206e3b41dd P ba2b6f8a3dac44a0ba159bafd5cc9bc7 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "cf70d9fd04894ea3812767cdb12156c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } attrs { replace: true } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.232192 13538 consensus_queue.cc:237] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } }
W20240503 08:04:50.256968 13510 consensus_peers.cc:485] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 -> Peer 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4:43125): Couldn't send request to peer 59a35ca031314d79a9b8676ac0a0fdde. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b6dbe71928443b09931a1b356044911. This is attempt 1: this message will repeat every 5th retry.
I20240503 08:04:50.265189 13616 raft_consensus.cc:1261] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.266005 13467 raft_consensus.cc:1261] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Refusing update from remote peer ba2b6f8a3dac44a0ba159bafd5cc9bc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 08:04:50.267086 14398 consensus_queue.cc:1035] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 08:04:50.268427 14426 consensus_queue.cc:1035] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 08:04:50.282451 14398 raft_consensus.cc:2932] T 4b6dbe71928443b09931a1b356044911 P ba2b6f8a3dac44a0ba159bafd5cc9bc7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.283068 13471 raft_consensus.cc:2932] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.289088 13614 raft_consensus.cc:2932] T 4b6dbe71928443b09931a1b356044911 P cf70d9fd04894ea3812767cdb12156c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.309808 13366 catalog_manager.cc:5484] T 4b6dbe71928443b09931a1b356044911 P f2462cc996fb441a847521d550b300a4 reported cstate change: config changed from index -1 to 2, NON_VOTER 59a35ca031314d79a9b8676ac0a0fdde (127.12.6.4) added. New cstate: current_term: 1 leader_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f2462cc996fb441a847521d550b300a4" member_type: VOTER last_known_addr { host: "127.12.6.1" port: 34427 } } peers { permanent_uuid: "cf70d9fd04894ea3812767cdb12156c5" member_type: VOTER last_known_addr { host: "127.12.6.3" port: 33827 } attrs { replace: true } } peers { permanent_uuid: "ba2b6f8a3dac44a0ba159bafd5cc9bc7" member_type: VOTER last_known_addr { host: "127.12.6.2" port: 44917 } } peers { permanent_uuid: "59a35ca031314d79a9b8676ac0a0fdde" member_type: NON_VOTER last_known_addr { host: "127.12.6.4" port: 43125 } attrs { promote: true } } }
I20240503 08:04:50.510900 14448 ts_tablet_manager.cc:902] T d4999c55f4034c3d9997841353a35afd P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
F20240503 08:04:50.512548 14448 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (init.queue_time_us, init.queue_time_us) (init.run_cpu_time_us, init.run_cpu_time_us) (init.run_wall_time_us, init.run_wall_time_us) (leader-initialization.queue_time_us, leader-initialization.queue_time_us) (leader-initialization.run_cpu_time_us, leader-initialization.run_cpu_time_us) (leader-initialization.run_wall_time_us, leader-initialization.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-008315f470c24029bd9271aad3c67a09.queue_time_us, tablet-download-pool-008315f470c24029bd9271aad3c67a09.queue_time_us) (tablet-download-pool-008315f470c24029bd9271aad3c67a09.run_cpu_time_us, tablet-download-pool-008315f470c24029bd9271aad3c67a09.run_cpu_time_us) (tablet-download-pool-008315f470c24029bd9271aad3c67a09.run_wall_time_us, tablet-download-pool-008315f470c24029bd9271aad3c67a09.run_wall_time_us) (tablet-download-pool-0786fc1af8ae401ebb5190aa117ad461.queue_time_us, tablet-download-pool-0786fc1af8ae401ebb5190aa117ad461.queue_time_us) (tablet-download-pool-0786fc1af8ae401ebb5190aa117ad461.run_cpu_time_us, tablet-download-pool-0786fc1af8ae401ebb5190aa117ad461.run_cpu_time_us) (tablet-download-pool-0786fc1af8ae401ebb5190aa117ad461.run_wall_time_us, tablet-download-pool-0786fc1af8ae401ebb5190aa117ad461.run_wall_time_us) (tablet-download-pool-0ef1acee9ffb4310bde01caa87d35754.queue_time_us, tablet-download-pool-0ef1acee9ffb4310bde01caa87d35754.queue_time_us) (tablet-download-pool-0ef1acee9ffb4310bde01caa87d35754.run_cpu_time_us, tablet-download-pool-0ef1acee9ffb4310bde01caa87d35754.run_cpu_time_us) (tablet-download-pool-0ef1acee9ffb4310bde01caa87d35754.run_wall_time_us, tablet-download-pool-0ef1acee9ffb4310bde01caa87d35754.run_wall_time_us) (tablet-download-pool-161465c92bbf4897bc9944c2b9869a9a.queue_time_us, tablet-download-pool-161465c92bbf4897bc9944c2b9869a9a.queue_time_us) (tablet-download-pool-161465c92bbf4897bc9944c2b9869a9a.run_cpu_time_us, tablet-download-pool-161465c92bbf4897bc9944c2b9869a9a.run_cpu_time_us) (tablet-download-pool-161465c92bbf4897bc9944c2b9869a9a.run_wall_time_us, tablet-download-pool-161465c92bbf4897bc9944c2b9869a9a.run_wall_time_us) (tablet-download-pool-3d88e8a1e1ad45f9a3cdf022a28162fb.queue_time_us, tablet-download-pool-3d88e8a1e1ad45f9a3cdf022a28162fb.queue_time_us) (tablet-download-pool-3d88e8a1e1ad45f9a3cdf022a28162fb.run_cpu_time_us, tablet-download-pool-3d88e8a1e1ad45f9a3cdf022a28162fb.run_cpu_time_us) (tablet-download-pool-3d88e8a1e1ad45f9a3cdf022a28162fb.run_wall_time_us, tablet-download-pool-3d88e8a1e1ad45f9a3cdf022a28162fb.run_wall_time_us) (tablet-download-pool-4c5d2ce341b749f8b2fd0c4ac23b4018.queue_time_us, tablet-download-pool-4c5d2ce341b749f8b2fd0c4ac23b4018.queue_time_us) (tablet-download-pool-4c5d2ce341b749f8b2fd0c4ac23b4018.run_cpu_time_us, tablet-download-pool-4c5d2ce341b749f8b2fd0c4ac23b4018.run_cpu_time_us) (tablet-download-pool-4c5d2ce341b749f8b2fd0c4ac23b4018.run_wall_time_us, tablet-download-pool-4c5d2ce341b749f8b2fd0c4ac23b4018.run_wall_time_us) (tablet-download-pool-768f7ed6c5cc40bd947c30cf5450c8f1.queue_time_us, tablet-download-pool-768f7ed6c5cc40bd947c30cf5450c8f1.queue_time_us) (tablet-download-pool-768f7ed6c5cc40bd947c30cf5450c8f1.run_cpu_time_us, tablet-download-pool-768f7ed6c5cc40bd947c30cf5450c8f1.run_cpu_time_us) (tablet-download-pool-768f7ed6c5cc40bd947c30cf5450c8f1.run_wall_time_us, tablet-download-pool-768f7ed6c5cc40bd947c30cf5450c8f1.run_wall_time_us) (tablet-download-pool-8497edff1aaf45e48da16ad73a649c6d.queue_time_us, tablet-download-pool-8497edff1aaf45e48da16ad73a649c6d.queue_time_us) (tablet-download-pool-8497edff1aaf45e48da16ad73a649c6d.run_cpu_time_us, tablet-download-pool-8497edff1aaf45e48da16ad73a649c6d.run_cpu_time_us) (tablet-download-pool-8497edff1aaf45e48da16ad73a649c6d.run_wall_time_us, tablet-download-pool-8497edff1aaf45e48da16ad73a649c6d.run_wall_time_us) (tablet-download-pool-aae93c20681b4e4fa9b521edbb9e65e0.queue_time_us, tablet-download-pool-aae93c20681b4e4fa9b521edbb9e65e0.queue_time_us) (tablet-download-pool-aae93c20681b4e4fa9b521edbb9e65e0.run_cpu_time_us, tablet-download-pool-aae93c20681b4e4fa9b521edbb9e65e0.run_cpu_time_us) (tablet-download-pool-aae93c20681b4e4fa9b521edbb9e65e0.run_wall_time_us, tablet-download-pool-aae93c20681b4e4fa9b521edbb9e65e0.run_wall_time_us) (tablet-download-pool-b8c70e75f946416cb5c23f68cfe59b98.queue_time_us, tablet-download-pool-b8c70e75f946416cb5c23f68cfe59b98.queue_time_us) (tablet-download-pool-b8c70e75f946416cb5c23f68cfe59b98.run_cpu_time_us, tablet-download-pool-b8c70e75f946416cb5c23f68cfe59b98.run_cpu_time_us) (tablet-download-pool-b8c70e75f946416cb5c23f68cfe59b98.run_wall_time_us, tablet-download-pool-b8c70e75f946416cb5c23f68cfe59b98.run_wall_time_us) (tablet-download-pool-d4999c55f4034c3d9997841353a35afd.queue_time_us, tablet-download-pool-d4999c55f4034c3d9997841353a35afd.queue_time_us) (tablet-download-pool-ef1e609185a44d14a5473954db083b1a.queue_time_us, tablet-download-pool-ef1e609185a44d14a5473954db083b1a.queue_time_us) (tablet-download-pool-ef1e609185a44d14a5473954db083b1a.run_cpu_time_us, tablet-download-pool-ef1e609185a44d14a5473954db083b1a.run_cpu_time_us) (tablet-download-pool-ef1e609185a44d14a5473954db083b1a.run_wall_time_us, tablet-download-pool-ef1e609185a44d14a5473954db083b1a.run_wall_time_us) (tablet-download-pool-f190cd50f6c2434ebf6d8e0d490eafe7.queue_time_us, tablet-download-pool-f190cd50f6c2434ebf6d8e0d490eafe7.queue_time_us) (tablet-download-pool-f190cd50f6c2434ebf6d8e0d490eafe7.run_cpu_time_us, tablet-download-pool-f190cd50f6c2434ebf6d8e0d490eafe7.run_cpu_time_us) (tablet-download-pool-f190cd50f6c2434ebf6d8e0d490eafe7.run_wall_time_us, tablet-download-pool-f190cd50f6c2434ebf6d8e0d490eafe7.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
*** Aborted at 1714723490 (unix time) try "date -d @1714723490" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800003018) received by PID 12312 (TID 0x7fe7fbd63700) from PID 12312; stack trace: ***
@ 0x2b0110 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b25f3 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7fe8bb0dd980 (unknown) at ??:0
@ 0x7fe8b4651fb7 gsignal at ??:0
@ 0x7fe8b4653921 abort at ??:0
@ 0x2b6106 abort at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7fe8b7447da4 google::LogMessage::SendToLog() at ??:0
I20240503 08:04:50.656576 14451 ts_tablet_manager.cc:902] T 6f02eda12b9544888367d6206e3b41dd P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer cf70d9fd04894ea3812767cdb12156c5 (127.12.6.3:33827)
@ 0x7fe8b7448950 google::LogMessage::Flush() at ??:0
@ 0x7fe8b744da8b google::LogMessageFatal::~LogMessageFatal() at ??:0
I20240503 08:04:50.703761 14455 ts_tablet_manager.cc:902] T 248eecaf8416442794c9558a2d73bdc9 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer f2462cc996fb441a847521d550b300a4 (127.12.6.1:34427)
@ 0x7fe8b7dd53c7 kudu::TraceMetrics::InternName() at ??:0
I20240503 08:04:50.723568 14457 ts_tablet_manager.cc:902] T 4b6dbe71928443b09931a1b356044911 P 59a35ca031314d79a9b8676ac0a0fdde: Initiating tablet copy from peer ba2b6f8a3dac44a0ba159bafd5cc9bc7 (127.12.6.2:44917)
@ 0x7fe8b7dab94f kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7fe8b7da90a5 kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7fe8bfbc9422 kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7fe8bfbcfc89 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7fe8bfc654ef kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7fe8bfc963cf kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7fe8bfc7c1ce kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
@ 0x7fe8bfc7c17a _ZNSt3__18__invokeIRZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBENS_8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEE4$_12JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSH_DpOSI_ at ??:0
@ 0x7fe8bfc7c10a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7fe8bfc7c0d2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7fe8bfc7b2fe std::__1::__function::__func<>::operator()() at ??:0
@ 0x7fe8c1021da5 std::__1::__function::__value_func<>::operator()() at ??:0
@ 0x7fe8c1021bda std::__1::function<>::operator()() at ??:0
@ 0x7fe8b7dad237 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe8b7db0692 kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe8b7db064a _ZNSt3__18__invokeIRZN4kudu10ThreadPool12CreateThreadEvE3$_2JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS5_DpOS6_ at ??:0
@ 0x7fe8b7db05da std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7fe8b7db05a2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7fe8b7daf89e std::__1::__function::__func<>::operator()() at ??:0
@ 0x7fe8c1021da5 std::__1::__function::__value_func<>::operator()() at ??:0
@ 0x7fe8c1021bda std::__1::function<>::operator()() at ??:0
@ 0x7fe8b7d8f937 kudu::Thread::SuperviseThread() at ??:0