[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240501 08:03:04.577824 16034 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.168.190:38685
I20240501 08:03:04.579088 16034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:03:04.580121 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:04.597994 16040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:04.598208 16043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:04.598946 16041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:04.599247 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:05.657158 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:05.657456 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:05.657640 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550585657617 us; error 0 us; skew 500 ppm
I20240501 08:03:05.658457 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:05.667676 16034 webserver.cc:460] Webserver started at http://127.15.168.190:33387/ using document root <none> and password file <none>
I20240501 08:03:05.668570 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:05.668817 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:05.669229 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:05.675904 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/master-0-root/instance:
uuid: "0d8f54a2926849e7aed86e245fa0e7a3"
format_stamp: "Formatted at 2024-05-01 08:03:05 on dist-test-slave-g8jl"
I20240501 08:03:05.683388 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20240501 08:03:05.688313 16049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:05.689360 16034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:03:05.689685 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/master-0-root
uuid: "0d8f54a2926849e7aed86e245fa0e7a3"
format_stamp: "Formatted at 2024-05-01 08:03:05 on dist-test-slave-g8jl"
I20240501 08:03:05.690030 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:05.747490 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:05.748989 16034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:03:05.749451 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:05.822714 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.190:38685
I20240501 08:03:05.822889 16100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.190:38685 every 8 connection(s)
I20240501 08:03:05.830612 16101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:05.856007 16101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3: Bootstrap starting.
I20240501 08:03:05.862169 16101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:05.864461 16101 log.cc:824] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3: Log is configured to *not* fsync() on all Append() calls
I20240501 08:03:05.868955 16101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3: No bootstrap required, opened a new log
I20240501 08:03:05.892854 16101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" member_type: VOTER }
I20240501 08:03:05.893463 16101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:05.893738 16101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8f54a2926849e7aed86e245fa0e7a3, State: Initialized, Role: FOLLOWER
I20240501 08:03:05.894537 16101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" member_type: VOTER }
I20240501 08:03:05.895051 16101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:03:05.895329 16101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:03:05.895656 16101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:05.900144 16101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" member_type: VOTER }
I20240501 08:03:05.900923 16101 leader_election.cc:302] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d8f54a2926849e7aed86e245fa0e7a3; no voters:
I20240501 08:03:05.902612 16101 leader_election.cc:288] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:03:05.902954 16104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:05.905757 16104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 1 LEADER]: Becoming Leader. State: Replica: 0d8f54a2926849e7aed86e245fa0e7a3, State: Running, Role: LEADER
I20240501 08:03:05.906622 16104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" member_type: VOTER }
I20240501 08:03:05.907764 16101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 08:03:05.917990 16106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d8f54a2926849e7aed86e245fa0e7a3. Latest consensus state: current_term: 1 leader_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" member_type: VOTER } }
I20240501 08:03:05.917996 16105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8f54a2926849e7aed86e245fa0e7a3" member_type: VOTER } }
I20240501 08:03:05.918826 16106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [sys.catalog]: This master's current role is: LEADER
I20240501 08:03:05.918884 16105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [sys.catalog]: This master's current role is: LEADER
I20240501 08:03:05.922959 16113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 08:03:05.935937 16113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 08:03:05.938311 16034 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240501 08:03:05.955417 16113 catalog_manager.cc:1347] Generated new cluster ID: 9980a66ac6554eaca4ce367792165a6b
I20240501 08:03:05.955683 16113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 08:03:05.976624 16113 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 08:03:05.978456 16113 catalog_manager.cc:1504] Loading token signing keys...
I20240501 08:03:06.010998 16113 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3: Generated new TSK 0
I20240501 08:03:06.012194 16113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 08:03:06.069837 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:06.080140 16123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:06.081259 16124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:06.085448 16034 server_base.cc:1034] running on GCE node
W20240501 08:03:06.086010 16126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:06.086762 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:06.086979 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:06.087175 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550586087156 us; error 0 us; skew 500 ppm
I20240501 08:03:06.087778 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:06.091765 16034 webserver.cc:460] Webserver started at http://127.15.168.129:41049/ using document root <none> and password file <none>
I20240501 08:03:06.092732 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:06.092931 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:06.093184 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:06.094332 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-0-root/instance:
uuid: "010248336543422c9bc66e7f458f20b8"
format_stamp: "Formatted at 2024-05-01 08:03:06 on dist-test-slave-g8jl"
I20240501 08:03:06.099766 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20240501 08:03:06.104604 16131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.105484 16034 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20240501 08:03:06.105871 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-0-root
uuid: "010248336543422c9bc66e7f458f20b8"
format_stamp: "Formatted at 2024-05-01 08:03:06 on dist-test-slave-g8jl"
I20240501 08:03:06.106284 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:06.134294 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:06.136085 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:06.138917 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:03:06.144903 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:03:06.145228 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.145596 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:03:06.145834 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.247373 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.129:35663
I20240501 08:03:06.247423 16193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.129:35663 every 8 connection(s)
I20240501 08:03:06.255731 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:06.269289 16198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:06.270510 16199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:06.280478 16194 heartbeater.cc:347] Connected to a master server at 127.15.168.190:38685
I20240501 08:03:06.281010 16194 heartbeater.cc:464] Registering TS with master...
I20240501 08:03:06.282532 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:06.282779 16194 heartbeater.cc:511] Master 127.15.168.190:38685 requested a full tablet report, sending...
W20240501 08:03:06.284634 16202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:06.285678 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20240501 08:03:06.285765 16066 ts_manager.cc:194] Registered new tserver with Master: 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663)
W20240501 08:03:06.285976 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:06.286206 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550586286190 us; error 0 us; skew 500 ppm
I20240501 08:03:06.287058 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:06.288111 16066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57474
I20240501 08:03:06.290321 16034 webserver.cc:460] Webserver started at http://127.15.168.130:36247/ using document root <none> and password file <none>
I20240501 08:03:06.291013 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:06.291260 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:06.291586 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:06.293049 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-1-root/instance:
uuid: "770c6387f34444969f41c80a4d187443"
format_stamp: "Formatted at 2024-05-01 08:03:06 on dist-test-slave-g8jl"
I20240501 08:03:06.300005 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20240501 08:03:06.304701 16206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.305980 16034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:03:06.306293 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-1-root
uuid: "770c6387f34444969f41c80a4d187443"
format_stamp: "Formatted at 2024-05-01 08:03:06 on dist-test-slave-g8jl"
I20240501 08:03:06.306643 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:06.336409 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:06.337744 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:06.340160 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:03:06.343184 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:03:06.343384 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.343606 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:03:06.343752 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.440538 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.130:46479
I20240501 08:03:06.440697 16268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.130:46479 every 8 connection(s)
I20240501 08:03:06.448380 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:06.461529 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:06.464623 16269 heartbeater.cc:347] Connected to a master server at 127.15.168.190:38685
I20240501 08:03:06.465045 16269 heartbeater.cc:464] Registering TS with master...
I20240501 08:03:06.466017 16269 heartbeater.cc:511] Master 127.15.168.190:38685 requested a full tablet report, sending...
I20240501 08:03:06.468096 16066 ts_manager.cc:194] Registered new tserver with Master: 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
W20240501 08:03:06.468508 16275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:06.469873 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:06.469923 16066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57482
I20240501 08:03:06.471378 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:06.472397 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:06.472648 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:06.472844 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550586472828 us; error 0 us; skew 500 ppm
I20240501 08:03:06.473618 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:06.476321 16034 webserver.cc:460] Webserver started at http://127.15.168.131:41085/ using document root <none> and password file <none>
I20240501 08:03:06.476922 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:06.477159 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:06.477502 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:06.478840 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-2-root/instance:
uuid: "a6195d8a074f45119b4b1852278933b0"
format_stamp: "Formatted at 2024-05-01 08:03:06 on dist-test-slave-g8jl"
I20240501 08:03:06.484428 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20240501 08:03:06.488730 16280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.489550 16034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:03:06.489878 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-2-root
uuid: "a6195d8a074f45119b4b1852278933b0"
format_stamp: "Formatted at 2024-05-01 08:03:06 on dist-test-slave-g8jl"
I20240501 08:03:06.490231 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714550584420650-16034-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:06.514376 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:06.515848 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:06.517427 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:03:06.520066 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:03:06.520327 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.520599 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:03:06.520800 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:06.610487 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.131:38397
I20240501 08:03:06.610544 16342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.131:38397 every 8 connection(s)
I20240501 08:03:06.628849 16343 heartbeater.cc:347] Connected to a master server at 127.15.168.190:38685
I20240501 08:03:06.629212 16343 heartbeater.cc:464] Registering TS with master...
I20240501 08:03:06.629966 16343 heartbeater.cc:511] Master 127.15.168.190:38685 requested a full tablet report, sending...
I20240501 08:03:06.631693 16066 ts_manager.cc:194] Registered new tserver with Master: a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:06.632460 16034 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0163792s
I20240501 08:03:06.633155 16066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57486
I20240501 08:03:06.636602 16034 test_util.cc:268] Using random seed: -357926619
I20240501 08:03:06.685770 16066 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57488:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240501 08:03:06.689780 16066 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240501 08:03:06.793090 16231 tablet_service.cc:1467] Processing CreateTablet for tablet a6ddf4eccea643e0ad9dab5f3f540762 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240501 08:03:06.793167 16232 tablet_service.cc:1467] Processing CreateTablet for tablet 3892dd57ad3c492a8a18f1daeaec8650 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240501 08:03:06.797976 16234 tablet_service.cc:1467] Processing CreateTablet for tablet 4054ebd7137b46709ec89d6bb4cb14ed (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240501 08:03:06.800257 16231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6ddf4eccea643e0ad9dab5f3f540762. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.799041 16227 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2acc2d44d4b90a4fc8a0feff2e14b (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240501 08:03:06.803113 16230 tablet_service.cc:1467] Processing CreateTablet for tablet 5f71969eb8734d88b9f5353402808317 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240501 08:03:06.804107 16227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2acc2d44d4b90a4fc8a0feff2e14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.805225 16229 tablet_service.cc:1467] Processing CreateTablet for tablet ef75ebb2bae845d1ba4cc4f6ed50b007 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240501 08:03:06.805387 16233 tablet_service.cc:1467] Processing CreateTablet for tablet 2b000bf3b40f47a89d5f7816a6e1bfed (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240501 08:03:06.807070 16228 tablet_service.cc:1467] Processing CreateTablet for tablet 9de2a8957d234de4b0291635a19561ae (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240501 08:03:06.809177 16226 tablet_service.cc:1467] Processing CreateTablet for tablet 4c083c711ad74a3bbacda9e85564abf6 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240501 08:03:06.810585 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4054ebd7137b46709ec89d6bb4cb14ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.811610 16230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f71969eb8734d88b9f5353402808317. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.813462 16233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b000bf3b40f47a89d5f7816a6e1bfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.814595 16226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c083c711ad74a3bbacda9e85564abf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.816345 16232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3892dd57ad3c492a8a18f1daeaec8650. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.817138 16228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de2a8957d234de4b0291635a19561ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.821231 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef75ebb2bae845d1ba4cc4f6ed50b007. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.852291 16306 tablet_service.cc:1467] Processing CreateTablet for tablet 3892dd57ad3c492a8a18f1daeaec8650 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240501 08:03:06.851166 16305 tablet_service.cc:1467] Processing CreateTablet for tablet a6ddf4eccea643e0ad9dab5f3f540762 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240501 08:03:06.853674 16308 tablet_service.cc:1467] Processing CreateTablet for tablet 4054ebd7137b46709ec89d6bb4cb14ed (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240501 08:03:06.856820 16303 tablet_service.cc:1467] Processing CreateTablet for tablet ef75ebb2bae845d1ba4cc4f6ed50b007 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240501 08:03:06.858186 16304 tablet_service.cc:1467] Processing CreateTablet for tablet 5f71969eb8734d88b9f5353402808317 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240501 08:03:06.869798 16300 tablet_service.cc:1467] Processing CreateTablet for tablet 4c083c711ad74a3bbacda9e85564abf6 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240501 08:03:06.865711 16307 tablet_service.cc:1467] Processing CreateTablet for tablet 2b000bf3b40f47a89d5f7816a6e1bfed (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240501 08:03:06.861483 16302 tablet_service.cc:1467] Processing CreateTablet for tablet 9de2a8957d234de4b0291635a19561ae (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240501 08:03:06.874573 16305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6ddf4eccea643e0ad9dab5f3f540762. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.863297 16301 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2acc2d44d4b90a4fc8a0feff2e14b (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240501 08:03:06.883641 16306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3892dd57ad3c492a8a18f1daeaec8650. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.886683 16300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c083c711ad74a3bbacda9e85564abf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.889461 16307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b000bf3b40f47a89d5f7816a6e1bfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.890954 16308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4054ebd7137b46709ec89d6bb4cb14ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.894362 16303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef75ebb2bae845d1ba4cc4f6ed50b007. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.895820 16304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f71969eb8734d88b9f5353402808317. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.899371 16302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de2a8957d234de4b0291635a19561ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.912122 16301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2acc2d44d4b90a4fc8a0feff2e14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.919641 16361 tablet_bootstrap.cc:492] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
W20240501 08:03:06.927603 16114 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:03:06.939548 16159 tablet_service.cc:1467] Processing CreateTablet for tablet 4054ebd7137b46709ec89d6bb4cb14ed (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240501 08:03:06.940186 16158 tablet_service.cc:1467] Processing CreateTablet for tablet 2b000bf3b40f47a89d5f7816a6e1bfed (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240501 08:03:06.942359 16159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4054ebd7137b46709ec89d6bb4cb14ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.947593 16158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b000bf3b40f47a89d5f7816a6e1bfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.950084 16156 tablet_service.cc:1467] Processing CreateTablet for tablet a6ddf4eccea643e0ad9dab5f3f540762 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240501 08:03:06.950712 16155 tablet_service.cc:1467] Processing CreateTablet for tablet 5f71969eb8734d88b9f5353402808317 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240501 08:03:06.949177 16157 tablet_service.cc:1467] Processing CreateTablet for tablet 3892dd57ad3c492a8a18f1daeaec8650 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240501 08:03:06.958312 16156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6ddf4eccea643e0ad9dab5f3f540762. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.962200 16157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3892dd57ad3c492a8a18f1daeaec8650. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.958281 16154 tablet_service.cc:1467] Processing CreateTablet for tablet ef75ebb2bae845d1ba4cc4f6ed50b007 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240501 08:03:06.964941 16153 tablet_service.cc:1467] Processing CreateTablet for tablet 9de2a8957d234de4b0291635a19561ae (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240501 08:03:06.967621 16155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f71969eb8734d88b9f5353402808317. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.969568 16154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef75ebb2bae845d1ba4cc4f6ed50b007. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.973558 16153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de2a8957d234de4b0291635a19561ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.972266 16151 tablet_service.cc:1467] Processing CreateTablet for tablet 4c083c711ad74a3bbacda9e85564abf6 (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240501 08:03:06.978510 16152 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2acc2d44d4b90a4fc8a0feff2e14b (DEFAULT_TABLE table=test-workload [id=cf4e1add53ce4e51a8efcc9bbbbcf533]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240501 08:03:06.981905 16151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c083c711ad74a3bbacda9e85564abf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:06.985117 16152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2acc2d44d4b90a4fc8a0feff2e14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:07.054551 16361 tablet_bootstrap.cc:654] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.085057 16362 tablet_bootstrap.cc:492] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.105574 16362 tablet_bootstrap.cc:654] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.106714 16363 tablet_bootstrap.cc:492] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.109313 16361 tablet_bootstrap.cc:492] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.110287 16361 ts_tablet_manager.cc:1367] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.197s user 0.013s sys 0.006s
I20240501 08:03:07.117175 16362 tablet_bootstrap.cc:492] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.117656 16362 ts_tablet_manager.cc:1367] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.061s user 0.011s sys 0.003s
I20240501 08:03:07.118340 16361 raft_consensus.cc:348] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.119146 16363 tablet_bootstrap.cc:654] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.119289 16361 raft_consensus.cc:374] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.119717 16361 raft_consensus.cc:729] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.120160 16362 raft_consensus.cc:348] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.121129 16362 raft_consensus.cc:374] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.120458 16361 consensus_queue.cc:260] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.121455 16362 raft_consensus.cc:729] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.122143 16362 consensus_queue.cc:260] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.125981 16269 heartbeater.cc:503] Master 127.15.168.190:38685 was elected leader, sending a full tablet report...
I20240501 08:03:07.126932 16361 ts_tablet_manager.cc:1398] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.016s user 0.010s sys 0.000s
I20240501 08:03:07.127969 16361 tablet_bootstrap.cc:492] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.129252 16362 ts_tablet_manager.cc:1398] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20240501 08:03:07.129675 16343 heartbeater.cc:503] Master 127.15.168.190:38685 was elected leader, sending a full tablet report...
I20240501 08:03:07.133667 16361 tablet_bootstrap.cc:654] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.137758 16362 tablet_bootstrap.cc:492] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.144948 16361 tablet_bootstrap.cc:492] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.145514 16361 ts_tablet_manager.cc:1367] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.003s
I20240501 08:03:07.148155 16361 raft_consensus.cc:348] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.148895 16362 tablet_bootstrap.cc:654] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.149051 16361 raft_consensus.cc:374] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.149499 16361 raft_consensus.cc:729] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.150242 16361 consensus_queue.cc:260] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.152510 16361 ts_tablet_manager.cc:1398] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240501 08:03:07.153410 16361 tablet_bootstrap.cc:492] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.155643 16362 tablet_bootstrap.cc:492] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.156090 16362 ts_tablet_manager.cc:1367] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.004s
I20240501 08:03:07.158653 16362 raft_consensus.cc:348] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.160521 16361 tablet_bootstrap.cc:654] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.162122 16363 tablet_bootstrap.cc:492] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.164546 16363 ts_tablet_manager.cc:1367] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.012s
I20240501 08:03:07.169710 16363 raft_consensus.cc:348] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.170440 16363 raft_consensus.cc:374] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.170727 16363 raft_consensus.cc:729] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.171911 16361 tablet_bootstrap.cc:492] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.159351 16362 raft_consensus.cc:374] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.171505 16363 consensus_queue.cc:260] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.172343 16362 raft_consensus.cc:729] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.172438 16361 ts_tablet_manager.cc:1367] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20240501 08:03:07.173082 16362 consensus_queue.cc:260] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.175500 16362 ts_tablet_manager.cc:1398] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20240501 08:03:07.175241 16361 raft_consensus.cc:348] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.175953 16361 raft_consensus.cc:374] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.176242 16361 raft_consensus.cc:729] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.176883 16362 tablet_bootstrap.cc:492] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.176882 16361 consensus_queue.cc:260] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.179493 16361 ts_tablet_manager.cc:1398] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240501 08:03:07.180380 16361 tablet_bootstrap.cc:492] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.183506 16362 tablet_bootstrap.cc:654] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.186379 16361 tablet_bootstrap.cc:654] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.198359 16361 tablet_bootstrap.cc:492] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
W20240501 08:03:07.199014 16270 tablet.cc:2369] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:03:07.201150 16361 ts_tablet_manager.cc:1367] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.004s
I20240501 08:03:07.203450 16363 ts_tablet_manager.cc:1398] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.038s user 0.006s sys 0.007s
I20240501 08:03:07.203925 16362 tablet_bootstrap.cc:492] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.204133 16194 heartbeater.cc:503] Master 127.15.168.190:38685 was elected leader, sending a full tablet report...
I20240501 08:03:07.204385 16363 tablet_bootstrap.cc:492] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.212416 16363 tablet_bootstrap.cc:654] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.217411 16362 ts_tablet_manager.cc:1367] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.026s
I20240501 08:03:07.222199 16362 raft_consensus.cc:348] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.223390 16362 raft_consensus.cc:374] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.223210 16361 raft_consensus.cc:348] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.223722 16362 raft_consensus.cc:729] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.224028 16361 raft_consensus.cc:374] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.224321 16361 raft_consensus.cc:729] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.224617 16362 consensus_queue.cc:260] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.224941 16361 consensus_queue.cc:260] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.227320 16361 ts_tablet_manager.cc:1398] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20240501 08:03:07.228219 16361 tablet_bootstrap.cc:492] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.229573 16366 raft_consensus.cc:482] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.230041 16366 raft_consensus.cc:504] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.231133 16362 ts_tablet_manager.cc:1398] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20240501 08:03:07.232033 16362 tablet_bootstrap.cc:492] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.232730 16366 leader_election.cc:288] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.222157 16363 tablet_bootstrap.cc:492] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.237336 16363 ts_tablet_manager.cc:1367] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.008s
I20240501 08:03:07.240561 16363 raft_consensus.cc:348] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.244892 16363 raft_consensus.cc:374] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.245738 16363 raft_consensus.cc:729] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.245465 16361 tablet_bootstrap.cc:654] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.249466 16362 tablet_bootstrap.cc:654] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.249989 16363 consensus_queue.cc:260] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.252399 16363 ts_tablet_manager.cc:1398] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20240501 08:03:07.253413 16363 tablet_bootstrap.cc:492] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.260819 16363 tablet_bootstrap.cc:654] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.272241 16361 tablet_bootstrap.cc:492] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.272848 16361 ts_tablet_manager.cc:1367] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.045s user 0.005s sys 0.013s
I20240501 08:03:07.274521 16362 tablet_bootstrap.cc:492] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.274966 16362 ts_tablet_manager.cc:1367] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.009s
I20240501 08:03:07.276190 16361 raft_consensus.cc:348] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.277014 16361 raft_consensus.cc:374] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.277392 16361 raft_consensus.cc:729] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.277927 16362 raft_consensus.cc:348] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.278110 16361 consensus_queue.cc:260] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.278668 16362 raft_consensus.cc:374] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.281440 16362 raft_consensus.cc:729] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.282101 16362 consensus_queue.cc:260] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.284143 16362 ts_tablet_manager.cc:1398] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20240501 08:03:07.284612 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.289731 16362 tablet_bootstrap.cc:492] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.292004 16361 ts_tablet_manager.cc:1398] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.019s user 0.003s sys 0.002s
I20240501 08:03:07.292981 16361 tablet_bootstrap.cc:492] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.295051 16366 raft_consensus.cc:482] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.295688 16366 raft_consensus.cc:504] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
W20240501 08:03:07.296092 16207 leader_election.cc:341] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.296422 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.297289 16318 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 0.
I20240501 08:03:07.298091 16366 leader_election.cc:288] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.298604 16207 leader_election.cc:302] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8
I20240501 08:03:07.299196 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
W20240501 08:03:07.300415 16207 leader_election.cc:341] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.301326 16366 raft_consensus.cc:2781] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.299980 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.301858 16366 raft_consensus.cc:482] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.302235 16366 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.304307 16362 tablet_bootstrap.cc:654] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
W20240501 08:03:07.304376 16207 leader_election.cc:341] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.305371 16207 leader_election.cc:302] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 770c6387f34444969f41c80a4d187443; no voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0
I20240501 08:03:07.310752 16363 tablet_bootstrap.cc:492] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.311201 16363 ts_tablet_manager.cc:1367] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.058s user 0.012s sys 0.006s
I20240501 08:03:07.313372 16361 tablet_bootstrap.cc:654] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.313906 16363 raft_consensus.cc:348] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.314720 16363 raft_consensus.cc:374] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.315060 16363 raft_consensus.cc:729] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.315843 16363 consensus_queue.cc:260] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.312950 16366 raft_consensus.cc:504] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.318373 16363 ts_tablet_manager.cc:1398] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 08:03:07.318825 16366 leader_election.cc:288] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.319336 16366 raft_consensus.cc:2726] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:07.320827 16363 tablet_bootstrap.cc:492] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.323211 16361 tablet_bootstrap.cc:492] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.323668 16361 ts_tablet_manager.cc:1367] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.010s
I20240501 08:03:07.326467 16361 raft_consensus.cc:348] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.327189 16361 raft_consensus.cc:374] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.327481 16361 raft_consensus.cc:729] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.328210 16361 consensus_queue.cc:260] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.329398 16362 tablet_bootstrap.cc:492] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.329833 16362 ts_tablet_manager.cc:1367] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.041s user 0.013s sys 0.004s
I20240501 08:03:07.331091 16361 ts_tablet_manager.cc:1398] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20240501 08:03:07.332207 16361 tablet_bootstrap.cc:492] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.333014 16362 raft_consensus.cc:348] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.333878 16362 raft_consensus.cc:374] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.334147 16362 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.334774 16362 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.335305 16363 tablet_bootstrap.cc:654] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.338330 16362 ts_tablet_manager.cc:1398] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.008s user 0.006s sys 0.002s
I20240501 08:03:07.339424 16362 tablet_bootstrap.cc:492] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.342146 16361 tablet_bootstrap.cc:654] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.348057 16363 tablet_bootstrap.cc:492] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.348246 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:07.348500 16363 ts_tablet_manager.cc:1367] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.001s
I20240501 08:03:07.348799 16318 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.352226 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
W20240501 08:03:07.353564 16207 leader_election.cc:341] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.354517 16363 raft_consensus.cc:348] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.355244 16363 raft_consensus.cc:374] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.355525 16363 raft_consensus.cc:729] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.356264 16363 consensus_queue.cc:260] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.358570 16363 ts_tablet_manager.cc:1398] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20240501 08:03:07.360183 16363 tablet_bootstrap.cc:492] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.362870 16362 tablet_bootstrap.cc:654] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.367615 16361 tablet_bootstrap.cc:492] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.367492 16318 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 1.
I20240501 08:03:07.369114 16361 ts_tablet_manager.cc:1367] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.005s
I20240501 08:03:07.368853 16207 leader_election.cc:302] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8
I20240501 08:03:07.370412 16381 raft_consensus.cc:2781] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.371799 16361 raft_consensus.cc:348] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.373289 16361 raft_consensus.cc:374] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.373788 16361 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.373484 16381 raft_consensus.cc:686] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:07.374534 16361 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.375627 16381 consensus_queue.cc:237] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.376950 16361 ts_tablet_manager.cc:1398] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240501 08:03:07.377857 16361 tablet_bootstrap.cc:492] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.381700 16363 tablet_bootstrap.cc:654] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.387568 16361 tablet_bootstrap.cc:654] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.392941 16362 tablet_bootstrap.cc:492] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.393417 16362 ts_tablet_manager.cc:1367] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.054s user 0.025s sys 0.007s
I20240501 08:03:07.395812 16362 raft_consensus.cc:348] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.397042 16362 raft_consensus.cc:374] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.397336 16362 raft_consensus.cc:729] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.398025 16362 consensus_queue.cc:260] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.400599 16362 ts_tablet_manager.cc:1398] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 08:03:07.402051 16362 tablet_bootstrap.cc:492] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.407994 16362 tablet_bootstrap.cc:654] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.407512 16064 catalog_manager.cc:5484] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 0 to 1, leader changed from <none> to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:07.411561 16363 tablet_bootstrap.cc:492] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.412067 16363 ts_tablet_manager.cc:1367] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.008s
I20240501 08:03:07.414858 16363 raft_consensus.cc:348] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.415616 16363 raft_consensus.cc:374] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.415906 16363 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.419286 16368 raft_consensus.cc:482] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.419955 16368 raft_consensus.cc:504] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.421216 16363 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.426299 16368 leader_election.cc:288] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.429852 16363 ts_tablet_manager.cc:1398] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.017s user 0.006s sys 0.002s
I20240501 08:03:07.431900 16362 tablet_bootstrap.cc:492] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.432372 16362 ts_tablet_manager.cc:1367] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.015s
I20240501 08:03:07.433090 16361 tablet_bootstrap.cc:492] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.433357 16363 tablet_bootstrap.cc:492] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.433553 16361 ts_tablet_manager.cc:1367] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.056s user 0.023s sys 0.002s
I20240501 08:03:07.434909 16362 raft_consensus.cc:348] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.435714 16362 raft_consensus.cc:374] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.435972 16361 raft_consensus.cc:348] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.436652 16361 raft_consensus.cc:374] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.436915 16361 raft_consensus.cc:729] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.440027 16361 consensus_queue.cc:260] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.435995 16362 raft_consensus.cc:729] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.444586 16368 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.444676 16362 consensus_queue.cc:260] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.445050 16368 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.445492 16361 ts_tablet_manager.cc:1398] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240501 08:03:07.449707 16361 tablet_bootstrap.cc:492] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443: Bootstrap starting.
I20240501 08:03:07.453878 16368 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663)
I20240501 08:03:07.466548 16361 tablet_bootstrap.cc:654] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.473137 16368 raft_consensus.cc:482] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.474722 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.475087 16368 raft_consensus.cc:504] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.476218 16169 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.476853 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.479743 16362 ts_tablet_manager.cc:1398] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.047s user 0.012s sys 0.008s
I20240501 08:03:07.480635 16362 tablet_bootstrap.cc:492] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.481858 16281 leader_election.cc:302] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.482530 16368 leader_election.cc:288] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
W20240501 08:03:07.483299 16281 leader_election.cc:341] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.483703 16397 raft_consensus.cc:2781] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.485323 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c083c711ad74a3bbacda9e85564abf6" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.486814 16168 raft_consensus.cc:2454] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.488684 16281 leader_election.cc:302] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.489101 16363 tablet_bootstrap.cc:654] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.487370 16397 raft_consensus.cc:482] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.489527 16397 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.490784 16368 raft_consensus.cc:2781] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.491333 16368 raft_consensus.cc:482] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.491667 16368 raft_consensus.cc:3037] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.506798 16397 raft_consensus.cc:504] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.509872 16397 leader_election.cc:288] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.512023 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.512992 16168 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.540480 16381 raft_consensus.cc:482] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.541056 16381 raft_consensus.cc:504] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.543866 16366 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.544379 16366 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.546248 16366 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397), 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663)
I20240501 08:03:07.548023 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.549387 16241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:07.552160 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.552497 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c083c711ad74a3bbacda9e85564abf6" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.548682 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.553983 16243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.556087 16318 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 0.
I20240501 08:03:07.565541 16243 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.555996 16362 tablet_bootstrap.cc:654] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.555361 16381 leader_election.cc:288] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.551651 16244 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.567808 16397 raft_consensus.cc:482] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240501 08:03:07.557785 16207 leader_election.cc:341] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.568238 16397 raft_consensus.cc:504] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.566581 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.570147 16283 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8
I20240501 08:03:07.567055 16368 raft_consensus.cc:504] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.573041 16168 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.573959 16368 leader_election.cc:288] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.574471 16368 raft_consensus.cc:2781] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.574352 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.574872 16368 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.569582 16207 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8
I20240501 08:03:07.555188 16242 raft_consensus.cc:2454] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.573011 16397 leader_election.cc:288] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.576375 16281 leader_election.cc:302] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters: 770c6387f34444969f41c80a4d187443
I20240501 08:03:07.576894 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.580137 16168 raft_consensus.cc:2454] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.580924 16363 tablet_bootstrap.cc:492] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.577088 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c083c711ad74a3bbacda9e85564abf6" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.581521 16167 raft_consensus.cc:3037] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.580931 16361 tablet_bootstrap.cc:492] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443: No bootstrap required, opened a new log
I20240501 08:03:07.582523 16361 ts_tablet_manager.cc:1367] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443: Time spent bootstrapping tablet: real 0.134s user 0.027s sys 0.017s
I20240501 08:03:07.584359 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c083c711ad74a3bbacda9e85564abf6" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:07.575240 16368 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.584991 16244 raft_consensus.cc:3037] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.585820 16363 ts_tablet_manager.cc:1367] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.153s user 0.018s sys 0.024s
I20240501 08:03:07.577960 16381 raft_consensus.cc:2781] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.586417 16381 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.586834 16381 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.581183 16281 leader_election.cc:302] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
W20240501 08:03:07.578398 16207 leader_election.cc:341] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.579674 16397 raft_consensus.cc:2781] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.588603 16363 raft_consensus.cc:348] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
W20240501 08:03:07.589563 16207 leader_election.cc:341] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.585465 16361 raft_consensus.cc:348] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.590564 16361 raft_consensus.cc:374] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.589521 16400 raft_consensus.cc:2781] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.591010 16400 raft_consensus.cc:482] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.591297 16400 raft_consensus.cc:3037] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.591689 16366 raft_consensus.cc:482] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.592414 16366 raft_consensus.cc:504] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.594312 16366 leader_election.cc:288] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.591816 16361 raft_consensus.cc:729] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.592048 16207 leader_election.cc:302] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 770c6387f34444969f41c80a4d187443; no voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0
I20240501 08:03:07.595284 16363 raft_consensus.cc:374] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.597982 16363 raft_consensus.cc:729] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.598080 16366 raft_consensus.cc:2726] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:07.598724 16363 consensus_queue.cc:260] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.602077 16244 raft_consensus.cc:2454] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.602890 16361 consensus_queue.cc:260] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.597592 16167 raft_consensus.cc:2454] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.595957 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.602082 16368 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.607969 16361 ts_tablet_manager.cc:1398] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20240501 08:03:07.607601 16283 leader_election.cc:302] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.611790 16366 raft_consensus.cc:482] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.612277 16366 raft_consensus.cc:504] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.614405 16366 leader_election.cc:288] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.615721 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.616307 16318 raft_consensus.cc:2454] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 0.
I20240501 08:03:07.604076 16397 raft_consensus.cc:686] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:07.618124 16397 consensus_queue.cc:237] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.620968 16400 raft_consensus.cc:504] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.606503 16242 raft_consensus.cc:2454] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.619601 16363 ts_tablet_manager.cc:1398] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.033s user 0.002s sys 0.006s
I20240501 08:03:07.626377 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.625970 16368 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663)
I20240501 08:03:07.626755 16381 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.627449 16363 tablet_bootstrap.cc:492] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.620635 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.627851 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
W20240501 08:03:07.629374 16281 leader_election.cc:341] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.630486 16168 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 0.
I20240501 08:03:07.633028 16372 raft_consensus.cc:482] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.633955 16372 raft_consensus.cc:504] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.637379 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.638986 16372 leader_election.cc:288] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.638384 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:07.641449 16207 leader_election.cc:302] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.635582 16401 raft_consensus.cc:2781] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.643182 16366 raft_consensus.cc:2781] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.643862 16366 raft_consensus.cc:482] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.644253 16366 raft_consensus.cc:3037] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.644028 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
W20240501 08:03:07.644804 16207 leader_election.cc:341] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.645131 16168 raft_consensus.cc:3037] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.645678 16207 leader_election.cc:302] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:07.649878 16400 leader_election.cc:288] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.651183 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:07.651763 16244 raft_consensus.cc:3037] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.651976 16381 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Requested vote from peers a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397), 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663)
I20240501 08:03:07.652500 16381 raft_consensus.cc:2781] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.652833 16381 raft_consensus.cc:482] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.653160 16381 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.643340 16402 raft_consensus.cc:482] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.654610 16401 raft_consensus.cc:686] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:07.659420 16168 raft_consensus.cc:2454] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.656910 16242 raft_consensus.cc:2379] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a6195d8a074f45119b4b1852278933b0 in current term 1: Already voted for candidate 770c6387f34444969f41c80a4d187443 in this term.
I20240501 08:03:07.660562 16281 leader_election.cc:302] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.671375 16400 raft_consensus.cc:2781] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.670287 16401 consensus_queue.cc:237] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.671624 16283 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443
I20240501 08:03:07.672241 16400 raft_consensus.cc:686] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:07.673277 16368 raft_consensus.cc:2726] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:03:07.673120 16400 consensus_queue.cc:237] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.673627 16366 raft_consensus.cc:504] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.674695 16244 raft_consensus.cc:2454] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.680447 16362 tablet_bootstrap.cc:492] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.656674 16402 raft_consensus.cc:504] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.686266 16362 ts_tablet_manager.cc:1367] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.206s user 0.018s sys 0.012s
W20240501 08:03:07.690773 16207 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.129:35663, user_credentials={real_user=slave}} blocked reactor thread for 45131us
I20240501 08:03:07.695559 16168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
W20240501 08:03:07.696408 16207 leader_election.cc:341] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:03:07.697163 16281 leader_election.cc:341] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.699043 16363 tablet_bootstrap.cc:654] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.695616 16402 leader_election.cc:288] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.698971 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.699463 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.695715 16362 raft_consensus.cc:348] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.695559 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.700178 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:07.700408 16366 leader_election.cc:288] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.701673 16362 raft_consensus.cc:374] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.703155 16318 raft_consensus.cc:2379] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 770c6387f34444969f41c80a4d187443 in current term 1: Already voted for candidate a6195d8a074f45119b4b1852278933b0 in this term.
I20240501 08:03:07.703317 16362 raft_consensus.cc:729] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.705634 16366 raft_consensus.cc:482] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.704913 16362 consensus_queue.cc:260] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.706091 16366 raft_consensus.cc:504] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.710593 16362 ts_tablet_manager.cc:1398] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.023s user 0.007s sys 0.001s
I20240501 08:03:07.709395 16244 raft_consensus.cc:2379] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6195d8a074f45119b4b1852278933b0 in current term 1: Already voted for candidate 770c6387f34444969f41c80a4d187443 in this term.
I20240501 08:03:07.712348 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:07.711678 16362 tablet_bootstrap.cc:492] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: Bootstrap starting.
I20240501 08:03:07.713219 16318 raft_consensus.cc:3037] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:03:07.717928 16207 leader_election.cc:341] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.716766 16381 raft_consensus.cc:504] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.714188 16366 leader_election.cc:288] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.722776 16283 leader_election.cc:302] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443
I20240501 08:03:07.723479 16066 catalog_manager.cc:5484] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
W20240501 08:03:07.727313 16207 leader_election.cc:341] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:07.729933 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.728888 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.731616 16317 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 0.
I20240501 08:03:07.731619 16167 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 0.
I20240501 08:03:07.732407 16318 raft_consensus.cc:2454] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 1.
I20240501 08:03:07.733268 16207 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 770c6387f34444969f41c80a4d187443; no voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0
I20240501 08:03:07.734344 16366 raft_consensus.cc:2726] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:03:07.734586 16207 leader_election.cc:302] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:07.734665 16401 raft_consensus.cc:2726] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:07.742900 16363 tablet_bootstrap.cc:492] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.743362 16363 ts_tablet_manager.cc:1367] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.116s user 0.010s sys 0.003s
I20240501 08:03:07.738400 16362 tablet_bootstrap.cc:654] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.735385 16381 leader_election.cc:288] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.740653 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.741904 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:07.745702 16167 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.739722 16366 raft_consensus.cc:2781] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.745934 16318 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.746219 16366 raft_consensus.cc:482] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.748492 16366 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.757645 16167 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 1.
I20240501 08:03:07.760074 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:07.740792 16207 leader_election.cc:302] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters: 010248336543422c9bc66e7f458f20b8
I20240501 08:03:07.761520 16381 raft_consensus.cc:2781] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.762140 16381 raft_consensus.cc:686] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:07.762917 16381 consensus_queue.cc:237] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.760399 16366 raft_consensus.cc:504] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.753489 16063 catalog_manager.cc:5484] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:07.769074 16207 leader_election.cc:302] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:07.771518 16407 raft_consensus.cc:2781] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.769614 16363 raft_consensus.cc:348] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.772670 16363 raft_consensus.cc:374] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.772966 16363 raft_consensus.cc:729] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.766415 16063 catalog_manager.cc:5484] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:07.779330 16362 tablet_bootstrap.cc:492] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: No bootstrap required, opened a new log
I20240501 08:03:07.780273 16362 ts_tablet_manager.cc:1367] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: Time spent bootstrapping tablet: real 0.069s user 0.016s sys 0.012s
I20240501 08:03:07.779474 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.772373 16366 leader_election.cc:288] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:07.780678 16167 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.782838 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.780946 16318 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 1.
I20240501 08:03:07.779747 16363 consensus_queue.cc:260] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.784292 16407 raft_consensus.cc:686] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:07.789357 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:07.790002 16318 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.790355 16167 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 1.
I20240501 08:03:07.789562 16407 consensus_queue.cc:237] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.793233 16362 raft_consensus.cc:348] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.793989 16362 raft_consensus.cc:374] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.794348 16362 raft_consensus.cc:729] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.794117 16066 catalog_manager.cc:5484] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 0 to 1, leader changed from <none> to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:07.791785 16207 leader_election.cc:302] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:07.795015 16362 consensus_queue.cc:260] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.798677 16414 raft_consensus.cc:2781] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.798967 16362 ts_tablet_manager.cc:1398] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: Time spent starting tablet: real 0.018s user 0.005s sys 0.001s
I20240501 08:03:07.799328 16401 raft_consensus.cc:482] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.807410 16401 raft_consensus.cc:504] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.799700 16363 ts_tablet_manager.cc:1398] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.056s user 0.007s sys 0.002s
I20240501 08:03:07.799422 16414 raft_consensus.cc:686] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:07.809427 16363 tablet_bootstrap.cc:492] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.812947 16414 consensus_queue.cc:237] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.817456 16401 leader_election.cc:288] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.818408 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8" is_pre_election: true
I20240501 08:03:07.818791 16363 tablet_bootstrap.cc:654] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.818799 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:07.819057 16167 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.819624 16366 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:03:07.819635 16242 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 0.
I20240501 08:03:07.821398 16281 leader_election.cc:302] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.822149 16401 raft_consensus.cc:2781] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:07.825626 16401 raft_consensus.cc:482] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:07.825981 16401 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.829465 16318 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 1.
I20240501 08:03:07.842293 16134 leader_election.cc:302] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 010248336543422c9bc66e7f458f20b8; no voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0
I20240501 08:03:07.843900 16372 raft_consensus.cc:2726] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:07.845628 16401 raft_consensus.cc:504] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.846891 16063 catalog_manager.cc:5484] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 0 to 1, leader changed from <none> to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:07.848417 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:07.849012 16167 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.850064 16401 leader_election.cc:288] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:07.848906 16063 catalog_manager.cc:5484] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 0 to 1, leader changed from <none> to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:07.853510 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:07.854125 16244 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.860638 16244 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.862067 16283 leader_election.cc:302] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:07.863656 16401 raft_consensus.cc:2781] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:07.864202 16401 raft_consensus.cc:686] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:07.865032 16401 consensus_queue.cc:237] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.870119 16167 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 1.
I20240501 08:03:07.871229 16363 tablet_bootstrap.cc:492] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.871806 16363 ts_tablet_manager.cc:1367] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.009s
I20240501 08:03:07.874483 16363 raft_consensus.cc:348] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.875284 16363 raft_consensus.cc:374] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.875587 16363 raft_consensus.cc:729] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.876207 16363 consensus_queue.cc:260] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:07.878494 16363 ts_tablet_manager.cc:1398] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20240501 08:03:07.879688 16363 tablet_bootstrap.cc:492] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: Bootstrap starting.
I20240501 08:03:07.880195 16167 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:07.888326 16366 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:03:07.891353 16363 tablet_bootstrap.cc:654] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:07.904109 16064 catalog_manager.cc:5484] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:07.922713 16363 tablet_bootstrap.cc:492] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: No bootstrap required, opened a new log
I20240501 08:03:07.923233 16363 ts_tablet_manager.cc:1367] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: Time spent bootstrapping tablet: real 0.044s user 0.012s sys 0.015s
I20240501 08:03:07.925729 16363 raft_consensus.cc:348] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.926457 16363 raft_consensus.cc:374] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:07.926753 16363 raft_consensus.cc:729] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Initialized, Role: FOLLOWER
I20240501 08:03:07.927340 16363 consensus_queue.cc:260] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.929272 16363 ts_tablet_manager.cc:1398] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
W20240501 08:03:07.929837 16114 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:03:07.996836 16372 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:07.997324 16372 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:07.998958 16372 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:08.000440 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:08.000244 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:08.001087 16317 raft_consensus.cc:2379] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 010248336543422c9bc66e7f458f20b8 in current term 1: Already voted for candidate a6195d8a074f45119b4b1852278933b0 in this term.
I20240501 08:03:08.001210 16244 raft_consensus.cc:2379] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 010248336543422c9bc66e7f458f20b8 in current term 1: Already voted for candidate 770c6387f34444969f41c80a4d187443 in this term.
I20240501 08:03:08.002187 16134 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 010248336543422c9bc66e7f458f20b8; no voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0
I20240501 08:03:08.002777 16372 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:08.007391 16372 raft_consensus.cc:2726] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:08.012987 16244 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:08.018883 16401 consensus_queue.cc:1035] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.029618 16402 consensus_queue.cc:1035] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.059675 16402 consensus_queue.cc:1035] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.069865 16400 consensus_queue.cc:1035] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:03:08.144531 16366 consensus_queue.cc:1035] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.154680 16167 raft_consensus.cc:3037] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:08.162446 16366 consensus_queue.cc:1035] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.209496 16366 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.222606 16381 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.240757 16400 consensus_queue.cc:1035] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.254431 16402 consensus_queue.cc:1035] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.277550 16381 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.289711 16366 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.418521 16402 consensus_queue.cc:1035] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:08.422479 16402 consensus_queue.cc:1035] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:03:08.931143 16114 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:03:09.743074 16439 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:09.743497 16439 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:09.744966 16439 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:09.745656 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443" is_pre_election: true
I20240501 08:03:09.745899 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0" is_pre_election: true
I20240501 08:03:09.746237 16244 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 1.
I20240501 08:03:09.746412 16317 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 1.
I20240501 08:03:09.747129 16134 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:09.747804 16439 raft_consensus.cc:2781] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240501 08:03:09.748075 16439 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:09.748324 16439 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:09.752383 16439 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:09.753718 16439 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:09.754359 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:09.754611 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:09.754793 16244 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:09.755074 16317 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:09.758824 16244 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:09.759218 16317 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:09.759646 16134 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:09.760239 16439 raft_consensus.cc:2781] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:09.761054 16439 raft_consensus.cc:686] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:09.761677 16439 consensus_queue.cc:237] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:09.767935 16064 catalog_manager.cc:5484] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 0 to 2, leader changed from <none> to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:10.155378 16317 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
mode: GRACEFUL
new_leader_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:46142
I20240501 08:03:10.155900 16317 raft_consensus.cc:595] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Received request to transfer leadership to TS 010248336543422c9bc66e7f458f20b8
I20240501 08:03:10.167192 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:55088
I20240501 08:03:10.167654 16242 raft_consensus.cc:595] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Received request to transfer leadership to TS 010248336543422c9bc66e7f458f20b8
I20240501 08:03:10.171584 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:55088
I20240501 08:03:10.172004 16242 raft_consensus.cc:595] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Received request to transfer leadership to TS 010248336543422c9bc66e7f458f20b8
I20240501 08:03:10.178022 16317 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:46142
I20240501 08:03:10.178483 16317 raft_consensus.cc:595] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:10.182188 16317 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:46142
I20240501 08:03:10.182610 16317 raft_consensus.cc:595] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:10.275784 16242 raft_consensus.cc:1261] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240501 08:03:10.276970 16451 consensus_queue.cc:1035] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:10.286536 16317 raft_consensus.cc:1261] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240501 08:03:10.288378 16453 consensus_queue.cc:1035] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:10.320283 16366 raft_consensus.cc:979] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: : Instructing follower 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:10.320710 16446 raft_consensus.cc:1067] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Signalling peer 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:10.322021 16169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:45554
I20240501 08:03:10.322479 16169 raft_consensus.cc:482] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:10.322743 16169 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.327015 16169 raft_consensus.cc:504] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.328500 16169 leader_election.cc:288] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:10.329200 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:10.329448 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:10.329855 16242 raft_consensus.cc:3032] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:10.329970 16317 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.330204 16242 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:10.330976 16242 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.332315 16242 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.336073 16317 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:10.337026 16132 leader_election.cc:302] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:10.337684 16453 raft_consensus.cc:2781] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:10.338094 16242 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:10.338140 16453 raft_consensus.cc:686] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:10.339159 16453 consensus_queue.cc:237] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.344972 16066 catalog_manager.cc:5484] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 1 to 2, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:10.370046 16400 raft_consensus.cc:979] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:10.370400 16400 raft_consensus.cc:1067] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:10.371803 16242 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48610
I20240501 08:03:10.372217 16242 raft_consensus.cc:482] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:10.372457 16242 raft_consensus.cc:3037] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.376546 16242 raft_consensus.cc:504] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.377903 16242 leader_election.cc:288] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 2 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:10.378715 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:10.378785 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de2a8957d234de4b0291635a19561ae" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:10.379411 16169 raft_consensus.cc:3037] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.379463 16317 raft_consensus.cc:3032] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:10.379920 16317 raft_consensus.cc:729] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:10.380692 16317 consensus_queue.cc:260] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.381925 16317 raft_consensus.cc:3037] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.385563 16169 raft_consensus.cc:2454] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 2.
I20240501 08:03:10.386605 16207 leader_election.cc:302] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:10.387230 16446 raft_consensus.cc:2781] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:10.387547 16317 raft_consensus.cc:2454] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 2.
I20240501 08:03:10.387713 16446 raft_consensus.cc:686] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:10.388541 16446 consensus_queue.cc:237] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.394812 16066 catalog_manager.cc:5484] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 1 to 2, leader changed from a6195d8a074f45119b4b1852278933b0 (127.15.168.131) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 2 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:10.505630 16366 raft_consensus.cc:979] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: : Instructing follower 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:10.505993 16446 raft_consensus.cc:1067] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Signalling peer 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:10.507285 16169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:45554
I20240501 08:03:10.507716 16169 raft_consensus.cc:482] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:10.508001 16169 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.512490 16169 raft_consensus.cc:504] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.513993 16169 leader_election.cc:288] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:10.514894 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:10.514948 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:10.515386 16242 raft_consensus.cc:3032] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:10.515422 16317 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.515722 16242 raft_consensus.cc:729] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:10.516383 16242 consensus_queue.cc:260] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.517364 16242 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.519927 16317 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:10.520787 16132 leader_election.cc:302] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:10.521572 16453 raft_consensus.cc:2781] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:10.521672 16242 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:10.521986 16453 raft_consensus.cc:686] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:10.522873 16453 consensus_queue.cc:237] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.529134 16066 catalog_manager.cc:5484] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 1 to 2, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:10.540758 16447 raft_consensus.cc:979] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:10.541115 16400 raft_consensus.cc:1067] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:10.542287 16242 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48610
I20240501 08:03:10.542717 16242 raft_consensus.cc:482] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:10.542982 16242 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.547029 16242 raft_consensus.cc:504] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.548408 16242 leader_election.cc:288] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 2 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:10.549134 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:10.549410 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:10.549672 16169 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.549872 16317 raft_consensus.cc:3032] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:10.550155 16317 raft_consensus.cc:729] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:10.550707 16317 consensus_queue.cc:260] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.551527 16317 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.553848 16169 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 2.
I20240501 08:03:10.554664 16207 leader_election.cc:302] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:10.555254 16446 raft_consensus.cc:2781] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:10.555500 16317 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 2.
I20240501 08:03:10.555594 16446 raft_consensus.cc:686] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:10.556434 16446 consensus_queue.cc:237] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.562525 16066 catalog_manager.cc:5484] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 1 to 2, leader changed from a6195d8a074f45119b4b1852278933b0 (127.15.168.131) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 2 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:10.577006 16447 raft_consensus.cc:979] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: : Instructing follower 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:10.577363 16400 raft_consensus.cc:1067] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Signalling peer 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:10.578536 16169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:45560
I20240501 08:03:10.578948 16169 raft_consensus.cc:482] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:10.579182 16169 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.583181 16169 raft_consensus.cc:504] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.584534 16169 leader_election.cc:288] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:10.585412 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:10.585412 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:10.586127 16317 raft_consensus.cc:3032] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:10.586187 16242 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.586571 16317 raft_consensus.cc:729] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:10.587320 16317 consensus_queue.cc:260] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.588511 16317 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:10.592458 16242 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:10.593487 16134 leader_election.cc:302] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:10.594116 16453 raft_consensus.cc:2781] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:10.594269 16317 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:10.594599 16453 raft_consensus.cc:686] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:10.595381 16453 consensus_queue.cc:237] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:10.601459 16066 catalog_manager.cc:5484] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 1 to 2, leader changed from a6195d8a074f45119b4b1852278933b0 (127.15.168.131) to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:10.768620 16317 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.769790 16453 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:10.778707 16242 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.780148 16467 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:10.901168 16317 raft_consensus.cc:1261] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.902288 16446 consensus_queue.cc:1035] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:10.911213 16169 raft_consensus.cc:1261] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.913226 16446 consensus_queue.cc:1035] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:03:10.939580 16169 raft_consensus.cc:1261] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.940665 16446 consensus_queue.cc:1035] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:10.949757 16317 raft_consensus.cc:1261] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.955058 16470 consensus_queue.cc:1035] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:10.966639 16242 raft_consensus.cc:1261] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.967873 16467 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:10.976310 16317 raft_consensus.cc:1261] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:10.977545 16453 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:11.066466 16317 raft_consensus.cc:1261] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:11.067571 16453 consensus_queue.cc:1035] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:11.075100 16242 raft_consensus.cc:1261] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:11.076628 16453 consensus_queue.cc:1035] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:13.207623 16034 auto_leader_rebalancer-test.cc:261] The leader distribution is
I20240501 08:03:13.208479 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:13.223088 16169 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
mode: GRACEFUL
new_leader_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:34596
I20240501 08:03:13.223598 16169 raft_consensus.cc:595] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Received request to transfer leadership to TS a6195d8a074f45119b4b1852278933b0
I20240501 08:03:13.232743 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5f71969eb8734d88b9f5353402808317"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:55094
I20240501 08:03:13.233214 16242 raft_consensus.cc:595] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Received request to transfer leadership to TS a6195d8a074f45119b4b1852278933b0
I20240501 08:03:13.234227 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240501 08:03:13.234638 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:13.330407 16483 raft_consensus.cc:979] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: : Instructing follower a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:13.330801 16453 raft_consensus.cc:1067] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Signalling peer a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:13.332036 16317 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:47144
I20240501 08:03:13.332943 16317 raft_consensus.cc:482] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:13.333226 16317 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:13.336930 16317 raft_consensus.cc:504] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:13.338258 16317 leader_election.cc:288] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 3 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:13.339104 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:13.339354 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:13.339594 16169 raft_consensus.cc:3032] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:13.339833 16242 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:13.339890 16169 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:13.340548 16169 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:13.341437 16169 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:13.344062 16242 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 3.
I20240501 08:03:13.344962 16283 leader_election.cc:302] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:13.345566 16400 raft_consensus.cc:2781] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:13.345997 16169 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 3.
I20240501 08:03:13.346552 16400 raft_consensus.cc:686] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:13.347357 16400 consensus_queue.cc:237] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:13.354358 16066 catalog_manager.cc:5484] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 2 to 3, leader changed from 010248336543422c9bc66e7f458f20b8 (127.15.168.129) to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 3 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:13.366019 16490 raft_consensus.cc:979] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: : Instructing follower a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:13.366385 16490 raft_consensus.cc:1067] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Signalling peer a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:13.367578 16317 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5f71969eb8734d88b9f5353402808317"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:47140
I20240501 08:03:13.367995 16317 raft_consensus.cc:482] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:13.368234 16317 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:13.372025 16317 raft_consensus.cc:504] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:13.373275 16317 leader_election.cc:288] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 2 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:13.373991 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:13.374155 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:13.374590 16169 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:13.374652 16242 raft_consensus.cc:3032] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:13.375030 16242 raft_consensus.cc:729] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:13.375767 16242 consensus_queue.cc:260] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:13.376951 16242 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:13.381395 16169 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 2.
I20240501 08:03:13.382378 16281 leader_election.cc:302] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:13.382887 16242 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 2.
I20240501 08:03:13.383100 16400 raft_consensus.cc:2781] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:13.383545 16400 raft_consensus.cc:686] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:13.384183 16400 consensus_queue.cc:237] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:13.390312 16066 catalog_manager.cc:5484] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 1 to 2, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 2 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:13.738282 16169 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:13.739346 16400 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:13.747706 16242 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:13.749678 16400 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:13.807898 16169 raft_consensus.cc:1261] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:13.809042 16400 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:13.817857 16242 raft_consensus.cc:1261] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:13.819304 16400 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:14.235420 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:14.240299 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:14.240656 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:15.241570 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:15.246441 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:15.246784 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:16.285053 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:16.298693 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:16.385046 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
W20240501 08:03:16.648414 16190 debug-util.cc:398] Leaking SignalData structure 0x7b08000ea060 after lost signal to thread 16037
W20240501 08:03:16.649148 16190 debug-util.cc:398] Leaking SignalData structure 0x7b08000ea740 after lost signal to thread 16100
W20240501 08:03:16.649863 16190 debug-util.cc:398] Leaking SignalData structure 0x7b08000405c0 after lost signal to thread 16193
W20240501 08:03:16.650502 16190 debug-util.cc:398] Leaking SignalData structure 0x7b08001db7e0 after lost signal to thread 16268
W20240501 08:03:16.651135 16190 debug-util.cc:398] Leaking SignalData structure 0x7b08001e73a0 after lost signal to thread 16342
I20240501 08:03:17.386044 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:17.390954 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:17.391350 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:18.392163 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:18.397142 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:18.397563 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:19.398280 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:19.403021 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:19.403393 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:20.404129 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:20.409468 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:20.409951 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:21.410699 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:21.415297 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:21.415664 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:22.416404 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:22.421113 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:22.421514 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:23.441691 16034 auto_leader_rebalancer-test.cc:280] The leader distribution is
I20240501 08:03:23.457535 16167 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:49800
I20240501 08:03:23.458324 16167 raft_consensus.cc:595] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:23.463289 16167 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:49800
I20240501 08:03:23.463728 16167 raft_consensus.cc:595] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:23.467840 16316 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:46142
I20240501 08:03:23.468290 16316 raft_consensus.cc:595] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:23.472600 16167 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:49800
I20240501 08:03:23.473071 16167 raft_consensus.cc:595] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:23.477048 16316 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5f71969eb8734d88b9f5353402808317"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
mode: GRACEFUL
new_leader_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:46142
I20240501 08:03:23.477548 16316 raft_consensus.cc:595] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 LEADER]: Received request to transfer leadership to TS 770c6387f34444969f41c80a4d187443
I20240501 08:03:23.485929 16453 raft_consensus.cc:979] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:23.486449 16515 raft_consensus.cc:1067] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:23.487869 16242 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48618
I20240501 08:03:23.488253 16242 raft_consensus.cc:482] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:23.488531 16242 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.048969 16242 raft_consensus.cc:504] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:24.052315 16242 leader_election.cc:288] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Requested vote from peers a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397), 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663)
W20240501 08:03:24.062582 16207 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.131:38397, user_credentials={real_user=slave}} blocked reactor thread for 99997.8us
I20240501 08:03:24.083137 16511 raft_consensus.cc:979] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.085805 16316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:24.086611 16316 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.070981 16517 raft_consensus.cc:979] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.094538 16516 raft_consensus.cc:1067] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.095191 16316 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.092293 16511 raft_consensus.cc:1067] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.099745 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054ebd7137b46709ec89d6bb4cb14ed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:24.100360 16169 raft_consensus.cc:3032] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:24.100657 16169 raft_consensus.cc:729] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:24.101368 16169 consensus_queue.cc:260] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:24.101930 16243 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48610
I20240501 08:03:24.102538 16243 raft_consensus.cc:482] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:24.102632 16169 raft_consensus.cc:3037] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.102931 16243 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:24.111521 16243 raft_consensus.cc:504] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.113793 16169 raft_consensus.cc:2454] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.121052 16243 leader_election.cc:288] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 4 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:24.121630 16242 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5f71969eb8734d88b9f5353402808317"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48610
I20240501 08:03:24.124320 16242 raft_consensus.cc:482] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:24.124954 16242 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.115739 16207 leader_election.cc:302] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:24.128713 16521 raft_consensus.cc:2781] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:24.129662 16521 raft_consensus.cc:686] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:24.129714 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:24.131392 16169 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:24.134431 16242 raft_consensus.cc:504] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.136364 16242 leader_election.cc:288] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:24.138437 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:24.139060 16167 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.142654 16169 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 4.
I20240501 08:03:24.144066 16521 consensus_queue.cc:237] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } }
I20240501 08:03:24.145680 16167 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.147011 16207 leader_election.cc:302] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:24.148010 16518 raft_consensus.cc:2781] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:24.148430 16518 raft_consensus.cc:686] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:24.149225 16518 consensus_queue.cc:237] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.150377 16207 leader_election.cc:302] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:24.160156 16518 raft_consensus.cc:2781] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:24.160727 16518 raft_consensus.cc:686] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:24.161624 16518 consensus_queue.cc:237] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.191676 16524 raft_consensus.cc:979] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.193770 16524 raft_consensus.cc:979] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: : Instructing follower 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.194221 16524 raft_consensus.cc:1067] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.195214 16530 raft_consensus.cc:1067] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Signalling peer 770c6387f34444969f41c80a4d187443 to start an election
I20240501 08:03:24.197614 16315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:24.198210 16315 raft_consensus.cc:3032] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:24.198496 16315 raft_consensus.cc:729] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:24.199239 16315 consensus_queue.cc:260] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.200364 16315 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.200603 16243 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48618
I20240501 08:03:24.200847 16242 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed"
dest_uuid: "770c6387f34444969f41c80a4d187443"
from {username='slave'} at 127.0.0.1:48618
I20240501 08:03:24.201009 16243 raft_consensus.cc:482] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:24.201201 16242 raft_consensus.cc:482] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:24.201308 16243 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.201562 16242 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.202800 16316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:24.203393 16316 raft_consensus.cc:3032] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:24.203891 16316 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:24.208019 16242 raft_consensus.cc:504] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.208937 16243 raft_consensus.cc:504] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.210395 16315 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.215461 16242 leader_election.cc:288] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:24.206604 16066 catalog_manager.cc:5484] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 2 to 3, leader changed from 010248336543422c9bc66e7f458f20b8 (127.15.168.129) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 3 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:24.215696 16243 leader_election.cc:288] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:24.210839 16316 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.217288 16316 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:24.217937 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:24.218459 16169 raft_consensus.cc:3032] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:24.218748 16169 raft_consensus.cc:729] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:24.217959 16167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:24.219336 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:24.219488 16167 raft_consensus.cc:3032] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:24.219863 16167 raft_consensus.cc:729] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:24.218720 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b000bf3b40f47a89d5f7816a6e1bfed" candidate_uuid: "770c6387f34444969f41c80a4d187443" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:24.219627 16169 consensus_queue.cc:260] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.220464 16318 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.220893 16169 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.220633 16167 consensus_queue.cc:260] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.219892 16317 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.222178 16167 raft_consensus.cc:3037] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:24.226655 16169 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.227799 16207 leader_election.cc:302] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:24.228713 16519 raft_consensus.cc:2781] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:24.229161 16519 raft_consensus.cc:686] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:24.229554 16167 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.229774 16318 raft_consensus.cc:2454] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.230007 16519 consensus_queue.cc:237] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.230772 16207 leader_election.cc:302] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:24.232221 16512 raft_consensus.cc:2781] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:24.232697 16512 raft_consensus.cc:686] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Becoming Leader. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:24.233192 16317 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 3.
I20240501 08:03:24.233569 16512 consensus_queue.cc:237] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:24.240492 16316 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 770c6387f34444969f41c80a4d187443 in term 4.
I20240501 08:03:24.254859 16066 catalog_manager.cc:5484] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 2 to 3, leader changed from a6195d8a074f45119b4b1852278933b0 (127.15.168.131) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 3 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:24.255986 16066 catalog_manager.cc:5484] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 3 to 4, leader changed from a6195d8a074f45119b4b1852278933b0 (127.15.168.131) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 4 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:24.257205 16066 catalog_manager.cc:5484] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 2 to 3, leader changed from 010248336543422c9bc66e7f458f20b8 (127.15.168.129) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 3 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:24.258533 16066 catalog_manager.cc:5484] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 reported cstate change: term changed from 2 to 3, leader changed from 010248336543422c9bc66e7f458f20b8 (127.15.168.129) to 770c6387f34444969f41c80a4d187443 (127.15.168.130). New cstate: current_term: 3 leader_uuid: "770c6387f34444969f41c80a4d187443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:24.567525 16316 raft_consensus.cc:1261] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20240501 08:03:24.568580 16519 consensus_queue.cc:1035] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:24.578159 16167 raft_consensus.cc:1261] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20240501 08:03:24.580083 16519 consensus_queue.cc:1035] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:24.668124 16169 raft_consensus.cc:1261] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:24.669507 16512 consensus_queue.cc:1035] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:03:24.680315 16316 raft_consensus.cc:1261] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:24.682118 16518 consensus_queue.cc:1035] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:03:24.690397 16169 raft_consensus.cc:1261] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:24.691598 16519 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:24.703783 16316 raft_consensus.cc:1261] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:24.705092 16512 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:24.726385 16316 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:24.727458 16518 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:24.728861 16316 raft_consensus.cc:1261] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:24.729997 16518 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:24.737424 16169 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:24.739161 16519 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:24.747035 16167 raft_consensus.cc:1261] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Refusing update from remote peer 770c6387f34444969f41c80a4d187443: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:24.748468 16512 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20240501 08:03:25.116984 16265 debug-util.cc:398] Leaking SignalData structure 0x7b0800131060 after lost signal to thread 16037
W20240501 08:03:25.117749 16265 debug-util.cc:398] Leaking SignalData structure 0x7b0800131740 after lost signal to thread 16100
W20240501 08:03:25.118628 16265 debug-util.cc:398] Leaking SignalData structure 0x7b08001d0dc0 after lost signal to thread 16193
W20240501 08:03:25.119406 16265 debug-util.cc:398] Leaking SignalData structure 0x7b080011b520 after lost signal to thread 16268
W20240501 08:03:25.120218 16265 debug-util.cc:398] Leaking SignalData structure 0x7b0800084ec0 after lost signal to thread 16342
I20240501 08:03:26.927639 16034 auto_leader_rebalancer-test.cc:295] The leader distribution is
I20240501 08:03:26.928295 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:26.935279 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:55094
I20240501 08:03:26.935854 16242 raft_consensus.cc:595] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 LEADER]: Received request to transfer leadership to TS 010248336543422c9bc66e7f458f20b8
I20240501 08:03:26.938189 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5f71969eb8734d88b9f5353402808317"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:55094
I20240501 08:03:26.938584 16242 raft_consensus.cc:595] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Received request to transfer leadership to TS 010248336543422c9bc66e7f458f20b8
I20240501 08:03:26.940501 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:55094
I20240501 08:03:26.941884 16242 raft_consensus.cc:595] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Received request to transfer leadership to TS a6195d8a074f45119b4b1852278933b0
I20240501 08:03:26.943851 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:55094
I20240501 08:03:26.944231 16242 raft_consensus.cc:595] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Received request to transfer leadership to TS a6195d8a074f45119b4b1852278933b0
I20240501 08:03:26.946156 16242 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b"
dest_uuid: "770c6387f34444969f41c80a4d187443"
mode: GRACEFUL
new_leader_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:55094
I20240501 08:03:26.946527 16242 raft_consensus.cc:595] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Received request to transfer leadership to TS 010248336543422c9bc66e7f458f20b8
I20240501 08:03:26.947392 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 5
I20240501 08:03:26.947806 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:26.970080 16552 raft_consensus.cc:979] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: : Instructing follower a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:26.970525 16553 raft_consensus.cc:1067] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Signalling peer a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:26.971928 16318 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:47140
I20240501 08:03:26.972366 16318 raft_consensus.cc:482] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:26.972683 16318 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:27.006982 16318 raft_consensus.cc:504] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.010162 16318 leader_election.cc:288] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 4 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:27.011262 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:27.011145 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6ddf4eccea643e0ad9dab5f3f540762" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:27.011958 16169 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:27.011914 16242 raft_consensus.cc:3032] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:27.012418 16242 raft_consensus.cc:729] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:27.013149 16242 consensus_queue.cc:260] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.014398 16242 raft_consensus.cc:3037] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:27.111812 16242 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 4.
I20240501 08:03:27.111812 16169 raft_consensus.cc:2454] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 4.
I20240501 08:03:27.113415 16283 leader_election.cc:302] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:27.115037 16551 raft_consensus.cc:2781] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:27.116165 16551 raft_consensus.cc:686] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 4 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:27.117256 16551 consensus_queue.cc:237] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.123955 16064 catalog_manager.cc:5484] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 3 to 4, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 4 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:27.256508 16552 raft_consensus.cc:979] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: : Instructing follower 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:27.256939 16553 raft_consensus.cc:1067] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Signalling peer 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:27.258332 16169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5f71969eb8734d88b9f5353402808317"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:45554
I20240501 08:03:27.258733 16169 raft_consensus.cc:482] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:27.258971 16169 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:27.316242 16169 raft_consensus.cc:504] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.318740 16169 leader_election.cc:288] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 4 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:27.319505 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:27.319586 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f71969eb8734d88b9f5353402808317" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:27.320175 16242 raft_consensus.cc:3032] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:27.320190 16318 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:27.320516 16242 raft_consensus.cc:729] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:27.321189 16242 consensus_queue.cc:260] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.322324 16242 raft_consensus.cc:3037] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:27.404304 16553 raft_consensus.cc:979] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: : Instructing follower 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:27.404697 16552 raft_consensus.cc:1067] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 LEADER]: Signalling peer 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:27.405866 16169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:45554
I20240501 08:03:27.406283 16169 raft_consensus.cc:482] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:27.406533 16169 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Advancing to term 5
I20240501 08:03:27.418768 16242 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 4.
I20240501 08:03:27.418768 16318 raft_consensus.cc:2454] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 4.
I20240501 08:03:27.418790 16169 raft_consensus.cc:504] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.420285 16134 leader_election.cc:302] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, 770c6387f34444969f41c80a4d187443; no voters:
I20240501 08:03:27.420801 16169 leader_election.cc:288] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 5 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:27.422106 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:27.422606 16560 raft_consensus.cc:2781] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:27.422698 16318 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Advancing to term 5
I20240501 08:03:27.423725 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3892dd57ad3c492a8a18f1daeaec8650" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:27.424033 16560 raft_consensus.cc:686] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 4 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:27.424324 16242 raft_consensus.cc:3032] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 LEADER]: Stepping down as leader of term 4
I20240501 08:03:27.424683 16242 raft_consensus.cc:729] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:27.424980 16560 consensus_queue.cc:237] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.425344 16242 consensus_queue.cc:260] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.426352 16242 raft_consensus.cc:3037] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Advancing to term 5
I20240501 08:03:27.435005 16064 catalog_manager.cc:5484] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 3 to 4, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 4 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:27.438431 16553 raft_consensus.cc:979] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: : Instructing follower 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:27.438822 16552 raft_consensus.cc:1067] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Signalling peer 010248336543422c9bc66e7f458f20b8 to start an election
I20240501 08:03:27.440027 16169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b"
dest_uuid: "010248336543422c9bc66e7f458f20b8"
from {username='slave'} at 127.0.0.1:45554
I20240501 08:03:27.440541 16169 raft_consensus.cc:482] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:27.440807 16169 raft_consensus.cc:3037] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:27.521945 16242 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 5.
I20240501 08:03:27.521945 16318 raft_consensus.cc:2454] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 5.
I20240501 08:03:27.521986 16169 raft_consensus.cc:504] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.523622 16132 leader_election.cc:302] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:27.523983 16169 leader_election.cc:288] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Requested vote from peers 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479), a6195d8a074f45119b4b1852278933b0 (127.15.168.131:38397)
I20240501 08:03:27.525113 16560 raft_consensus.cc:2781] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 5 FOLLOWER]: Leader election won for term 5
I20240501 08:03:27.525583 16560 raft_consensus.cc:686] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 5 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:27.525698 16318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6195d8a074f45119b4b1852278933b0"
I20240501 08:03:27.526011 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2acc2d44d4b90a4fc8a0feff2e14b" candidate_uuid: "010248336543422c9bc66e7f458f20b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:27.526276 16318 raft_consensus.cc:3037] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:27.526721 16242 raft_consensus.cc:3032] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:27.526450 16560 consensus_queue.cc:237] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.527070 16242 raft_consensus.cc:729] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:27.527709 16242 consensus_queue.cc:260] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.528683 16242 raft_consensus.cc:3037] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:27.534013 16064 catalog_manager.cc:5484] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 4 to 5, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 5 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:27.544220 16243 raft_consensus.cc:1261] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:27.545185 16551 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:27.549610 16553 raft_consensus.cc:979] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: : Instructing follower a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:27.550083 16552 raft_consensus.cc:1067] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Signalling peer a6195d8a074f45119b4b1852278933b0 to start an election
I20240501 08:03:27.551410 16316 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007"
dest_uuid: "a6195d8a074f45119b4b1852278933b0"
from {username='slave'} at 127.0.0.1:47140
I20240501 08:03:27.551918 16316 raft_consensus.cc:482] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:27.552282 16316 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:27.554229 16169 raft_consensus.cc:1261] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:27.555353 16551 consensus_queue.cc:1035] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:27.627149 16242 raft_consensus.cc:2454] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:27.627149 16318 raft_consensus.cc:2454] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010248336543422c9bc66e7f458f20b8 in term 2.
I20240501 08:03:27.628839 16132 leader_election.cc:302] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010248336543422c9bc66e7f458f20b8, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:27.629761 16560 raft_consensus.cc:2781] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:27.630193 16560 raft_consensus.cc:686] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Becoming Leader. State: Replica: 010248336543422c9bc66e7f458f20b8, State: Running, Role: LEADER
I20240501 08:03:27.630975 16560 consensus_queue.cc:237] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.638239 16064 catalog_manager.cc:5484] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 reported cstate change: term changed from 1 to 2, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to 010248336543422c9bc66e7f458f20b8 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "010248336543422c9bc66e7f458f20b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:27.729393 16316 raft_consensus.cc:504] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.731824 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "010248336543422c9bc66e7f458f20b8"
I20240501 08:03:27.731992 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef75ebb2bae845d1ba4cc4f6ed50b007" candidate_uuid: "a6195d8a074f45119b4b1852278933b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "770c6387f34444969f41c80a4d187443"
I20240501 08:03:27.732486 16242 raft_consensus.cc:3032] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:27.732547 16316 leader_election.cc:288] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 3 election: Requested vote from peers 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663), 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479)
I20240501 08:03:27.732903 16242 raft_consensus.cc:729] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 770c6387f34444969f41c80a4d187443, State: Running, Role: LEADER
I20240501 08:03:27.732481 16169 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:27.733608 16242 consensus_queue.cc:260] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.734731 16242 raft_consensus.cc:3037] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:27.812789 16316 raft_consensus.cc:1261] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:27.813938 16560 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:27.821774 16243 raft_consensus.cc:1261] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:27.823159 16561 consensus_queue.cc:1035] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:27.832104 16169 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 3.
I20240501 08:03:27.832106 16242 raft_consensus.cc:2454] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6195d8a074f45119b4b1852278933b0 in term 3.
I20240501 08:03:27.833307 16283 leader_election.cc:302] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770c6387f34444969f41c80a4d187443, a6195d8a074f45119b4b1852278933b0; no voters:
I20240501 08:03:27.834062 16551 raft_consensus.cc:2781] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:27.834415 16551 raft_consensus.cc:686] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Becoming Leader. State: Replica: a6195d8a074f45119b4b1852278933b0, State: Running, Role: LEADER
I20240501 08:03:27.835106 16551 consensus_queue.cc:237] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } }
I20240501 08:03:27.841763 16063 catalog_manager.cc:5484] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 reported cstate change: term changed from 2 to 3, leader changed from 770c6387f34444969f41c80a4d187443 (127.15.168.130) to a6195d8a074f45119b4b1852278933b0 (127.15.168.131). New cstate: current_term: 3 leader_uuid: "a6195d8a074f45119b4b1852278933b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:27.948881 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:27.953733 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:27.954138 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:27.997196 16242 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 5 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240501 08:03:27.998200 16561 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:03:28.005681 16316 raft_consensus.cc:1261] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 5 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240501 08:03:28.007668 16560 consensus_queue.cc:1035] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:03:28.134496 16316 raft_consensus.cc:1261] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:28.136085 16560 consensus_queue.cc:1035] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6195d8a074f45119b4b1852278933b0" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:28.139236 16243 raft_consensus.cc:1261] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Refusing update from remote peer 010248336543422c9bc66e7f458f20b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:28.140784 16571 consensus_queue.cc:1035] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:28.251389 16243 raft_consensus.cc:1261] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:28.252599 16554 consensus_queue.cc:1035] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770c6387f34444969f41c80a4d187443" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:28.262544 16169 raft_consensus.cc:1261] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Refusing update from remote peer a6195d8a074f45119b4b1852278933b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:28.264039 16551 consensus_queue.cc:1035] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010248336543422c9bc66e7f458f20b8" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:28.954903 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:28.960238 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:28.960620 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:29.961431 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:29.966498 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:29.966828 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:30.967463 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:30.972448 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:30.972826 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:31.973527 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:31.978091 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:31.978459 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:32.979156 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:32.984140 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:32.984532 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:33.985316 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:33.990145 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:33.990540 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:34.991602 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:34.997990 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:34.998379 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:35.999198 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:36.003970 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:36.004442 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:37.024309 16034 auto_leader_rebalancer-test.cc:306] The leader distribution is
I20240501 08:03:37.024904 16034 tablet_server.cc:178] TabletServer@127.15.168.129:0 shutting down...
I20240501 08:03:37.049211 16034 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240501 08:03:37.049844 16034 tablet_replica.cc:330] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.050570 16034 raft_consensus.cc:2229] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.051185 16034 raft_consensus.cc:2258] T 4054ebd7137b46709ec89d6bb4cb14ed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.053512 16034 tablet_replica.cc:330] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.054037 16034 raft_consensus.cc:2229] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.054416 16034 raft_consensus.cc:2258] T a6ddf4eccea643e0ad9dab5f3f540762 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.055918 16034 tablet_replica.cc:330] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.056388 16034 raft_consensus.cc:2229] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.056752 16034 raft_consensus.cc:2258] T 2b000bf3b40f47a89d5f7816a6e1bfed P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.058480 16034 tablet_replica.cc:330] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.059011 16034 raft_consensus.cc:2229] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 2 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.059989 16034 raft_consensus.cc:2258] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.061728 16034 tablet_replica.cc:330] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.062136 16034 raft_consensus.cc:2229] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.062486 16034 raft_consensus.cc:2258] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 010248336543422c9bc66e7f458f20b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.063876 16034 tablet_replica.cc:330] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.064481 16034 raft_consensus.cc:2229] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 4 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.065491 16034 raft_consensus.cc:2258] T 5f71969eb8734d88b9f5353402808317 P 010248336543422c9bc66e7f458f20b8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.067351 16034 tablet_replica.cc:330] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.067770 16034 raft_consensus.cc:2229] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.068104 16034 raft_consensus.cc:2258] T 9de2a8957d234de4b0291635a19561ae P 010248336543422c9bc66e7f458f20b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.069638 16034 tablet_replica.cc:330] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.070024 16034 raft_consensus.cc:2229] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 5 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.070724 16034 raft_consensus.cc:2258] T 3892dd57ad3c492a8a18f1daeaec8650 P 010248336543422c9bc66e7f458f20b8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.072095 16034 tablet_replica.cc:330] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8: stopping tablet replica
I20240501 08:03:37.072469 16034 raft_consensus.cc:2229] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.072772 16034 raft_consensus.cc:2258] T 4c083c711ad74a3bbacda9e85564abf6 P 010248336543422c9bc66e7f458f20b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240501 08:03:37.148001 16281 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.168.129:35663: connect: Connection refused (error 111)
W20240501 08:03:37.151662 16281 consensus_peers.cc:485] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 -> Peer 010248336543422c9bc66e7f458f20b8 (127.15.168.129:35663): Couldn't send request to peer 010248336543422c9bc66e7f458f20b8. Status: Network error: Client connection negotiation failed: client connection to 127.15.168.129:35663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 08:03:37.167117 16034 tablet_server.cc:194] TabletServer@127.15.168.129:0 shutdown complete.
I20240501 08:03:37.182814 16034 tablet_server.cc:178] TabletServer@127.15.168.130:0 shutting down...
I20240501 08:03:37.205583 16034 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240501 08:03:37.206158 16034 tablet_replica.cc:330] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.206748 16034 raft_consensus.cc:2229] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 5 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.207248 16034 raft_consensus.cc:2258] T 3892dd57ad3c492a8a18f1daeaec8650 P 770c6387f34444969f41c80a4d187443 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.208850 16034 tablet_replica.cc:330] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.209348 16034 raft_consensus.cc:2229] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.209708 16034 raft_consensus.cc:2258] T 4c083c711ad74a3bbacda9e85564abf6 P 770c6387f34444969f41c80a4d187443 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.211182 16034 tablet_replica.cc:330] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.211628 16034 raft_consensus.cc:2229] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.212308 16034 raft_consensus.cc:2258] T 2b000bf3b40f47a89d5f7816a6e1bfed P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.213850 16034 tablet_replica.cc:330] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.214347 16034 raft_consensus.cc:2229] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 2 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.215010 16034 raft_consensus.cc:2258] T 9de2a8957d234de4b0291635a19561ae P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.216603 16034 tablet_replica.cc:330] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.217020 16034 raft_consensus.cc:2229] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.217397 16034 raft_consensus.cc:2258] T 5f71969eb8734d88b9f5353402808317 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.218994 16034 tablet_replica.cc:330] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.219398 16034 raft_consensus.cc:2229] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.219750 16034 raft_consensus.cc:2258] T ef75ebb2bae845d1ba4cc4f6ed50b007 P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.221557 16034 tablet_replica.cc:330] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.222159 16034 raft_consensus.cc:2229] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.222687 16034 raft_consensus.cc:2258] T f2f2acc2d44d4b90a4fc8a0feff2e14b P 770c6387f34444969f41c80a4d187443 [term 2 FOLLOWER]: Raft consensus is shut down!
W20240501 08:03:37.224530 16283 consensus_peers.cc:485] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 -> Peer 770c6387f34444969f41c80a4d187443 (127.15.168.130:46479): Couldn't send request to peer 770c6387f34444969f41c80a4d187443. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:03:37.224601 16034 tablet_replica.cc:330] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.225189 16034 raft_consensus.cc:2229] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.225592 16034 raft_consensus.cc:2258] T a6ddf4eccea643e0ad9dab5f3f540762 P 770c6387f34444969f41c80a4d187443 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.227066 16034 tablet_replica.cc:330] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443: stopping tablet replica
I20240501 08:03:37.227478 16034 raft_consensus.cc:2229] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 3 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.228106 16034 raft_consensus.cc:2258] T 4054ebd7137b46709ec89d6bb4cb14ed P 770c6387f34444969f41c80a4d187443 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.269323 16034 tablet_server.cc:194] TabletServer@127.15.168.130:0 shutdown complete.
I20240501 08:03:37.283838 16034 tablet_server.cc:178] TabletServer@127.15.168.131:0 shutting down...
I20240501 08:03:37.307700 16034 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240501 08:03:37.308262 16034 tablet_replica.cc:330] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.308810 16034 raft_consensus.cc:2229] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.309358 16034 raft_consensus.cc:2258] T 3892dd57ad3c492a8a18f1daeaec8650 P a6195d8a074f45119b4b1852278933b0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.310994 16034 tablet_replica.cc:330] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.311453 16034 raft_consensus.cc:2229] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.311844 16034 raft_consensus.cc:2258] T 2b000bf3b40f47a89d5f7816a6e1bfed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.313386 16034 tablet_replica.cc:330] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.313822 16034 raft_consensus.cc:2229] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 3 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.314539 16034 raft_consensus.cc:2258] T ef75ebb2bae845d1ba4cc4f6ed50b007 P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.316107 16034 tablet_replica.cc:330] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.316668 16034 raft_consensus.cc:2229] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.317045 16034 raft_consensus.cc:2258] T 5f71969eb8734d88b9f5353402808317 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.318521 16034 tablet_replica.cc:330] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.318961 16034 raft_consensus.cc:2229] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.319336 16034 raft_consensus.cc:2258] T 9de2a8957d234de4b0291635a19561ae P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.320816 16034 tablet_replica.cc:330] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.321261 16034 raft_consensus.cc:2229] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.321641 16034 raft_consensus.cc:2258] T f2f2acc2d44d4b90a4fc8a0feff2e14b P a6195d8a074f45119b4b1852278933b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.323045 16034 tablet_replica.cc:330] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.323464 16034 raft_consensus.cc:2229] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 1 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.324132 16034 raft_consensus.cc:2258] T 4c083c711ad74a3bbacda9e85564abf6 P a6195d8a074f45119b4b1852278933b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.325593 16034 tablet_replica.cc:330] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.325985 16034 raft_consensus.cc:2229] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:03:37.326332 16034 raft_consensus.cc:2258] T 4054ebd7137b46709ec89d6bb4cb14ed P a6195d8a074f45119b4b1852278933b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.327752 16034 tablet_replica.cc:330] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0: stopping tablet replica
I20240501 08:03:37.328178 16034 raft_consensus.cc:2229] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 4 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.328828 16034 raft_consensus.cc:2258] T a6ddf4eccea643e0ad9dab5f3f540762 P a6195d8a074f45119b4b1852278933b0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.355410 16034 tablet_server.cc:194] TabletServer@127.15.168.131:0 shutdown complete.
I20240501 08:03:37.370539 16034 master.cc:537] Master@127.15.168.190:38685 shutting down...
I20240501 08:03:37.561524 16034 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 1 LEADER]: Raft consensus shutting down.
I20240501 08:03:37.562129 16034 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:03:37.562446 16034 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0d8f54a2926849e7aed86e245fa0e7a3: stopping tablet replica
I20240501 08:03:37.579859 16034 master.cc:552] Master@127.15.168.190:38685 shutdown complete.
010248336543422c9bc66e7f458f20b8 4
770c6387f34444969f41c80a4d187443 4
a6195d8a074f45119b4b1852278933b0 1
010248336543422c9bc66e7f458f20b8 3
770c6387f34444969f41c80a4d187443 3
a6195d8a074f45119b4b1852278933b0 3
010248336543422c9bc66e7f458f20b8 0
770c6387f34444969f41c80a4d187443 8
a6195d8a074f45119b4b1852278933b0 1
010248336543422c9bc66e7f458f20b8 3
770c6387f34444969f41c80a4d187443 3
a6195d8a074f45119b4b1852278933b0 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (33041 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20240501 08:03:37.615804 16034 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.168.190:40777
I20240501 08:03:37.616845 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:37.622664 16590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:37.623243 16591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:37.624083 16593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:37.624814 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:37.625744 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:37.625950 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:37.626091 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550617626075 us; error 0 us; skew 500 ppm
I20240501 08:03:37.626554 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:37.633018 16034 webserver.cc:460] Webserver started at http://127.15.168.190:33701/ using document root <none> and password file <none>
I20240501 08:03:37.633504 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:37.633675 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:37.633922 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:37.635035 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/master-0-root/instance:
uuid: "e0b8e2c6c3a04323b71acd16e17c744c"
format_stamp: "Formatted at 2024-05-01 08:03:37 on dist-test-slave-g8jl"
I20240501 08:03:37.639338 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240501 08:03:37.642277 16598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.642995 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 08:03:37.643234 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/master-0-root
uuid: "e0b8e2c6c3a04323b71acd16e17c744c"
format_stamp: "Formatted at 2024-05-01 08:03:37 on dist-test-slave-g8jl"
I20240501 08:03:37.643464 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:37.676218 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:37.677379 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:37.711712 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.190:40777
I20240501 08:03:37.711807 16649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.190:40777 every 8 connection(s)
I20240501 08:03:37.715528 16650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:37.724601 16650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c: Bootstrap starting.
I20240501 08:03:37.729164 16650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:37.732857 16650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c: No bootstrap required, opened a new log
I20240501 08:03:37.734731 16650 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" member_type: VOTER }
I20240501 08:03:37.735172 16650 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:37.735373 16650 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b8e2c6c3a04323b71acd16e17c744c, State: Initialized, Role: FOLLOWER
I20240501 08:03:37.735852 16650 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" member_type: VOTER }
I20240501 08:03:37.736282 16650 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:03:37.736490 16650 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:03:37.736738 16650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:37.740272 16650 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" member_type: VOTER }
I20240501 08:03:37.740726 16650 leader_election.cc:302] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0b8e2c6c3a04323b71acd16e17c744c; no voters:
I20240501 08:03:37.741742 16650 leader_election.cc:288] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:03:37.742061 16653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:37.743193 16653 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 1 LEADER]: Becoming Leader. State: Replica: e0b8e2c6c3a04323b71acd16e17c744c, State: Running, Role: LEADER
I20240501 08:03:37.743955 16653 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" member_type: VOTER }
I20240501 08:03:37.744504 16650 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [sys.catalog]: configured and running, proceeding with master startup.
I20240501 08:03:37.749594 16654 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" member_type: VOTER } }
I20240501 08:03:37.750147 16654 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [sys.catalog]: This master's current role is: LEADER
I20240501 08:03:37.751195 16655 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b8e2c6c3a04323b71acd16e17c744c. Latest consensus state: current_term: 1 leader_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b8e2c6c3a04323b71acd16e17c744c" member_type: VOTER } }
I20240501 08:03:37.751718 16655 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [sys.catalog]: This master's current role is: LEADER
I20240501 08:03:37.752045 16660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 08:03:37.757792 16660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 08:03:37.758729 16034 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240501 08:03:37.765605 16660 catalog_manager.cc:1347] Generated new cluster ID: 172b3873a9e34c51b37f6eaf5831ef36
I20240501 08:03:37.765808 16660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 08:03:37.800376 16660 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 08:03:37.801549 16660 catalog_manager.cc:1504] Loading token signing keys...
I20240501 08:03:37.816269 16660 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c: Generated new TSK 0
I20240501 08:03:37.816766 16660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 08:03:37.824540 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:37.830858 16671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:37.831382 16672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:37.832762 16674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:37.833518 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:37.834285 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:37.834465 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:37.834585 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550617834574 us; error 0 us; skew 500 ppm
I20240501 08:03:37.835058 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:37.837261 16034 webserver.cc:460] Webserver started at http://127.15.168.129:41051/ using document root <none> and password file <none>
I20240501 08:03:37.837723 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:37.837874 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:37.838076 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:37.839053 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-0-root/instance:
uuid: "fb35cf97ff15428fa584058ace6b6875"
format_stamp: "Formatted at 2024-05-01 08:03:37 on dist-test-slave-g8jl"
I20240501 08:03:37.843370 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240501 08:03:37.846576 16679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.847259 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 08:03:37.847533 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-0-root
uuid: "fb35cf97ff15428fa584058ace6b6875"
format_stamp: "Formatted at 2024-05-01 08:03:37 on dist-test-slave-g8jl"
I20240501 08:03:37.847795 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:37.869450 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:37.870594 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:37.871960 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:03:37.874051 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:03:37.874244 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.874459 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:03:37.874614 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.910427 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.129:42471
I20240501 08:03:37.910511 16741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.129:42471 every 8 connection(s)
I20240501 08:03:37.914475 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:37.921960 16746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:37.922437 16747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:37.923873 16749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:37.926102 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:37.926574 16742 heartbeater.cc:347] Connected to a master server at 127.15.168.190:40777
I20240501 08:03:37.926887 16742 heartbeater.cc:464] Registering TS with master...
I20240501 08:03:37.926975 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:37.927203 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:37.927348 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550617927332 us; error 0 us; skew 500 ppm
I20240501 08:03:37.927500 16742 heartbeater.cc:511] Master 127.15.168.190:40777 requested a full tablet report, sending...
I20240501 08:03:37.927897 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:37.929486 16615 ts_manager.cc:194] Registered new tserver with Master: fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471)
I20240501 08:03:37.930496 16034 webserver.cc:460] Webserver started at http://127.15.168.130:41599/ using document root <none> and password file <none>
I20240501 08:03:37.930994 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:37.931169 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:37.931430 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:37.931376 16615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35158
I20240501 08:03:37.932778 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-1-root/instance:
uuid: "947186496a094f8885f3ad358cf8d03f"
format_stamp: "Formatted at 2024-05-01 08:03:37 on dist-test-slave-g8jl"
I20240501 08:03:37.937032 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20240501 08:03:37.940004 16754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.940657 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:03:37.940907 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-1-root
uuid: "947186496a094f8885f3ad358cf8d03f"
format_stamp: "Formatted at 2024-05-01 08:03:37 on dist-test-slave-g8jl"
I20240501 08:03:37.941176 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:37.950176 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:37.951232 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:37.952458 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:03:37.954512 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:03:37.954694 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.954905 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:03:37.955049 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:37.990242 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.130:40167
I20240501 08:03:37.990329 16816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.130:40167 every 8 connection(s)
I20240501 08:03:37.994617 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:03:38.001372 16820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:38.001897 16821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:03:38.004352 16823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:03:38.005645 16034 server_base.cc:1034] running on GCE node
I20240501 08:03:38.006078 16817 heartbeater.cc:347] Connected to a master server at 127.15.168.190:40777
I20240501 08:03:38.006470 16817 heartbeater.cc:464] Registering TS with master...
I20240501 08:03:38.006539 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:03:38.006844 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:03:38.007047 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550618007023 us; error 0 us; skew 500 ppm
I20240501 08:03:38.007356 16817 heartbeater.cc:511] Master 127.15.168.190:40777 requested a full tablet report, sending...
I20240501 08:03:38.007822 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:03:38.009524 16615 ts_manager.cc:194] Registered new tserver with Master: 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.010547 16034 webserver.cc:460] Webserver started at http://127.15.168.131:41737/ using document root <none> and password file <none>
I20240501 08:03:38.011186 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:03:38.011251 16615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35166
I20240501 08:03:38.011473 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:03:38.011813 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:03:38.013265 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-2-root/instance:
uuid: "125c40043a7044939023b53e34e3a7e7"
format_stamp: "Formatted at 2024-05-01 08:03:38 on dist-test-slave-g8jl"
I20240501 08:03:38.017382 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240501 08:03:38.020378 16828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:38.021046 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:03:38.021322 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-2-root
uuid: "125c40043a7044939023b53e34e3a7e7"
format_stamp: "Formatted at 2024-05-01 08:03:38 on dist-test-slave-g8jl"
I20240501 08:03:38.021562 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714550584420650-16034-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:03:38.033425 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:03:38.034529 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:03:38.036020 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:03:38.038074 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:03:38.038245 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:38.038463 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:03:38.038602 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:03:38.075353 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.131:34079
I20240501 08:03:38.075430 16890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.131:34079 every 8 connection(s)
I20240501 08:03:38.087374 16891 heartbeater.cc:347] Connected to a master server at 127.15.168.190:40777
I20240501 08:03:38.087711 16891 heartbeater.cc:464] Registering TS with master...
I20240501 08:03:38.088352 16891 heartbeater.cc:511] Master 127.15.168.190:40777 requested a full tablet report, sending...
I20240501 08:03:38.090071 16615 ts_manager.cc:194] Registered new tserver with Master: 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.090204 16034 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.012112809s
I20240501 08:03:38.091729 16615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35172
I20240501 08:03:38.093075 16034 test_util.cc:268] Using random seed: -326470145
I20240501 08:03:38.112668 16615 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35188:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240501 08:03:38.115335 16615 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240501 08:03:38.191953 16707 tablet_service.cc:1467] Processing CreateTablet for tablet fd7076d607314b098f3f1152781ae2f3 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240501 08:03:38.194480 16705 tablet_service.cc:1467] Processing CreateTablet for tablet f67ff0cac9ea438196146004f9f9d740 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240501 08:03:38.194696 16856 tablet_service.cc:1467] Processing CreateTablet for tablet fd7076d607314b098f3f1152781ae2f3 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240501 08:03:38.194808 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 9caf16c8ff3e4249b54bda04211f369d (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240501 08:03:38.195811 16856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd7076d607314b098f3f1152781ae2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.196960 16706 tablet_service.cc:1467] Processing CreateTablet for tablet 364c7a4942e24357bc5446f5d6ef7b01 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240501 08:03:38.198204 16853 tablet_service.cc:1467] Processing CreateTablet for tablet afd455adc27f480e97b44f042a946db3 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240501 08:03:38.198374 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 321032c3a694400e820755e4cadd5815 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240501 08:03:38.199465 16853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd455adc27f480e97b44f042a946db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.200040 16782 tablet_service.cc:1467] Processing CreateTablet for tablet fd7076d607314b098f3f1152781ae2f3 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240501 08:03:38.200585 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 96373142392848c5b375f1f89cf2ec86 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240501 08:03:38.201259 16782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd7076d607314b098f3f1152781ae2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.201792 16852 tablet_service.cc:1467] Processing CreateTablet for tablet d937a5aa00c74349a0953014727fdf2f (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240501 08:03:38.203002 16852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d937a5aa00c74349a0953014727fdf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.202718 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 44feef2d33b349bdb3cd2ae0389a5745 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240501 08:03:38.198382 16704 tablet_service.cc:1467] Processing CreateTablet for tablet afd455adc27f480e97b44f042a946db3 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240501 08:03:38.206669 16851 tablet_service.cc:1467] Processing CreateTablet for tablet 321032c3a694400e820755e4cadd5815 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240501 08:03:38.207003 16781 tablet_service.cc:1467] Processing CreateTablet for tablet 364c7a4942e24357bc5446f5d6ef7b01 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240501 08:03:38.207901 16851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 321032c3a694400e820755e4cadd5815. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.208218 16781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364c7a4942e24357bc5446f5d6ef7b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.210371 16849 tablet_service.cc:1467] Processing CreateTablet for tablet 44feef2d33b349bdb3cd2ae0389a5745 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240501 08:03:38.211544 16849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44feef2d33b349bdb3cd2ae0389a5745. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.212589 16780 tablet_service.cc:1467] Processing CreateTablet for tablet f67ff0cac9ea438196146004f9f9d740 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240501 08:03:38.213816 16780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f67ff0cac9ea438196146004f9f9d740. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.200603 16703 tablet_service.cc:1467] Processing CreateTablet for tablet d937a5aa00c74349a0953014727fdf2f (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240501 08:03:38.205176 16850 tablet_service.cc:1467] Processing CreateTablet for tablet 96373142392848c5b375f1f89cf2ec86 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240501 08:03:38.216307 16850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96373142392848c5b375f1f89cf2ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.220636 16779 tablet_service.cc:1467] Processing CreateTablet for tablet afd455adc27f480e97b44f042a946db3 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240501 08:03:38.199486 16854 tablet_service.cc:1467] Processing CreateTablet for tablet f67ff0cac9ea438196146004f9f9d740 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240501 08:03:38.222084 16854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f67ff0cac9ea438196146004f9f9d740. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.222350 16778 tablet_service.cc:1467] Processing CreateTablet for tablet d937a5aa00c74349a0953014727fdf2f (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240501 08:03:38.223639 16778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d937a5aa00c74349a0953014727fdf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.196698 16855 tablet_service.cc:1467] Processing CreateTablet for tablet 364c7a4942e24357bc5446f5d6ef7b01 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240501 08:03:38.226107 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364c7a4942e24357bc5446f5d6ef7b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.226716 16707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd7076d607314b098f3f1152781ae2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.227381 16703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d937a5aa00c74349a0953014727fdf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.227677 16776 tablet_service.cc:1467] Processing CreateTablet for tablet 96373142392848c5b375f1f89cf2ec86 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240501 08:03:38.228866 16776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96373142392848c5b375f1f89cf2ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.228957 16702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 321032c3a694400e820755e4cadd5815. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.230217 16706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364c7a4942e24357bc5446f5d6ef7b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.230643 16775 tablet_service.cc:1467] Processing CreateTablet for tablet 44feef2d33b349bdb3cd2ae0389a5745 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240501 08:03:38.231524 16700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44feef2d33b349bdb3cd2ae0389a5745. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.231855 16775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44feef2d33b349bdb3cd2ae0389a5745. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.235580 16779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd455adc27f480e97b44f042a946db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.226357 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 321032c3a694400e820755e4cadd5815 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240501 08:03:38.237924 16777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 321032c3a694400e820755e4cadd5815. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.234486 16705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f67ff0cac9ea438196146004f9f9d740. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.242137 16699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9caf16c8ff3e4249b54bda04211f369d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.248847 16704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd455adc27f480e97b44f042a946db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.251717 16701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96373142392848c5b375f1f89cf2ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.291397 16774 tablet_service.cc:1467] Processing CreateTablet for tablet 9caf16c8ff3e4249b54bda04211f369d (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240501 08:03:38.292675 16774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9caf16c8ff3e4249b54bda04211f369d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.307821 16906 tablet_bootstrap.cc:492] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.327462 16854 tablet_service.cc:1467] Processing CreateTablet for tablet 9caf16c8ff3e4249b54bda04211f369d (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240501 08:03:38.328763 16854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9caf16c8ff3e4249b54bda04211f369d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.366873 16906 tablet_bootstrap.cc:654] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.367986 16907 tablet_bootstrap.cc:492] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.370126 16707 tablet_service.cc:1467] Processing CreateTablet for tablet 478b286082d34eecac6cc50fdcbe5e19 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240501 08:03:38.371654 16707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 478b286082d34eecac6cc50fdcbe5e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.384815 16907 tablet_bootstrap.cc:654] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.386826 16908 tablet_bootstrap.cc:492] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.391875 16856 tablet_service.cc:1467] Processing CreateTablet for tablet 478b286082d34eecac6cc50fdcbe5e19 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240501 08:03:38.393071 16856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 478b286082d34eecac6cc50fdcbe5e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.393189 16775 tablet_service.cc:1467] Processing CreateTablet for tablet 478b286082d34eecac6cc50fdcbe5e19 (DEFAULT_TABLE table=test-workload [id=3dce463dea5b4e8f825030113b608814]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240501 08:03:38.395856 16906 tablet_bootstrap.cc:492] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.396349 16906 ts_tablet_manager.cc:1367] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.089s user 0.013s sys 0.012s
I20240501 08:03:38.398866 16906 raft_consensus.cc:348] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.399590 16906 raft_consensus.cc:374] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.399865 16906 raft_consensus.cc:729] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.400507 16906 consensus_queue.cc:260] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.404465 16908 tablet_bootstrap.cc:654] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.407647 16775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 478b286082d34eecac6cc50fdcbe5e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:03:38.430938 16906 ts_tablet_manager.cc:1398] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.034s user 0.011s sys 0.015s
I20240501 08:03:38.431795 16891 heartbeater.cc:503] Master 127.15.168.190:40777 was elected leader, sending a full tablet report...
I20240501 08:03:38.432657 16908 tablet_bootstrap.cc:492] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.432816 16906 tablet_bootstrap.cc:492] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.433109 16908 ts_tablet_manager.cc:1367] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.023s
I20240501 08:03:38.435813 16907 tablet_bootstrap.cc:492] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.436343 16907 ts_tablet_manager.cc:1367] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.069s user 0.015s sys 0.020s
I20240501 08:03:38.436527 16908 raft_consensus.cc:348] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.437085 16908 raft_consensus.cc:374] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.437332 16908 raft_consensus.cc:729] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.438057 16908 consensus_queue.cc:260] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.438889 16907 raft_consensus.cc:348] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.439600 16907 raft_consensus.cc:374] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.439702 16906 tablet_bootstrap.cc:654] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.439927 16907 raft_consensus.cc:729] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.440621 16907 consensus_queue.cc:260] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.447546 16908 ts_tablet_manager.cc:1398] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.014s user 0.009s sys 0.002s
I20240501 08:03:38.448345 16817 heartbeater.cc:503] Master 127.15.168.190:40777 was elected leader, sending a full tablet report...
I20240501 08:03:38.458242 16742 heartbeater.cc:503] Master 127.15.168.190:40777 was elected leader, sending a full tablet report...
I20240501 08:03:38.458786 16907 ts_tablet_manager.cc:1398] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.022s user 0.018s sys 0.003s
I20240501 08:03:38.459690 16907 tablet_bootstrap.cc:492] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.465445 16907 tablet_bootstrap.cc:654] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.465886 16908 tablet_bootstrap.cc:492] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.468385 16906 tablet_bootstrap.cc:492] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.468901 16906 ts_tablet_manager.cc:1367] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.013s
I20240501 08:03:38.471500 16906 raft_consensus.cc:348] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.472318 16906 raft_consensus.cc:374] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.472661 16906 raft_consensus.cc:729] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.473661 16908 tablet_bootstrap.cc:654] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.473477 16906 consensus_queue.cc:260] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.478125 16906 ts_tablet_manager.cc:1398] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20240501 08:03:38.480048 16906 tablet_bootstrap.cc:492] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.484768 16907 tablet_bootstrap.cc:492] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.485275 16907 ts_tablet_manager.cc:1367] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.007s
I20240501 08:03:38.485786 16906 tablet_bootstrap.cc:654] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.487764 16907 raft_consensus.cc:348] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.488497 16907 raft_consensus.cc:374] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.488780 16907 raft_consensus.cc:729] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.489765 16908 tablet_bootstrap.cc:492] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.489445 16907 consensus_queue.cc:260] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.490200 16908 ts_tablet_manager.cc:1367] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.001s
I20240501 08:03:38.491758 16907 ts_tablet_manager.cc:1398] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:03:38.492594 16907 tablet_bootstrap.cc:492] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.492723 16908 raft_consensus.cc:348] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.493461 16908 raft_consensus.cc:374] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.493742 16908 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
W20240501 08:03:38.494021 16818 tablet.cc:2369] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:03:38.494423 16908 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.496697 16908 ts_tablet_manager.cc:1398] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:03:38.497634 16908 tablet_bootstrap.cc:492] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.499540 16907 tablet_bootstrap.cc:654] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.503029 16908 tablet_bootstrap.cc:654] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.510989 16907 tablet_bootstrap.cc:492] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.511473 16907 ts_tablet_manager.cc:1367] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.012s
I20240501 08:03:38.513787 16906 tablet_bootstrap.cc:492] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.514253 16906 ts_tablet_manager.cc:1367] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.034s user 0.015s sys 0.012s
I20240501 08:03:38.514081 16907 raft_consensus.cc:348] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.514736 16907 raft_consensus.cc:374] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.515043 16907 raft_consensus.cc:729] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.515730 16908 tablet_bootstrap.cc:492] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.515689 16907 consensus_queue.cc:260] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.516229 16908 ts_tablet_manager.cc:1367] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.006s
I20240501 08:03:38.516299 16906 raft_consensus.cc:348] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.517033 16906 raft_consensus.cc:374] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.517325 16906 raft_consensus.cc:729] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.517918 16907 ts_tablet_manager.cc:1398] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20240501 08:03:38.517987 16906 consensus_queue.cc:260] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.518810 16907 tablet_bootstrap.cc:492] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.518962 16908 raft_consensus.cc:348] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.519631 16908 raft_consensus.cc:374] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.519894 16908 raft_consensus.cc:729] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.520583 16908 consensus_queue.cc:260] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.524418 16907 tablet_bootstrap.cc:654] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.526480 16908 ts_tablet_manager.cc:1398] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240501 08:03:38.527406 16908 tablet_bootstrap.cc:492] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.529556 16906 ts_tablet_manager.cc:1398] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20240501 08:03:38.533047 16908 tablet_bootstrap.cc:654] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.534765 16907 tablet_bootstrap.cc:492] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.535147 16907 ts_tablet_manager.cc:1367] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20240501 08:03:38.535868 16906 tablet_bootstrap.cc:492] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.537652 16908 tablet_bootstrap.cc:492] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.538208 16908 ts_tablet_manager.cc:1367] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20240501 08:03:38.537657 16907 raft_consensus.cc:348] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.538568 16907 raft_consensus.cc:374] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.538885 16907 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.539573 16907 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.540277 16908 raft_consensus.cc:348] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.541217 16908 raft_consensus.cc:374] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.541509 16908 raft_consensus.cc:729] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.541712 16907 ts_tablet_manager.cc:1398] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20240501 08:03:38.541870 16906 tablet_bootstrap.cc:654] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.542721 16907 tablet_bootstrap.cc:492] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.542309 16908 consensus_queue.cc:260] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.546620 16908 ts_tablet_manager.cc:1398] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20240501 08:03:38.547495 16908 tablet_bootstrap.cc:492] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.548696 16907 tablet_bootstrap.cc:654] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.553030 16908 tablet_bootstrap.cc:654] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.564069 16908 tablet_bootstrap.cc:492] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.564508 16908 ts_tablet_manager.cc:1367] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20240501 08:03:38.566689 16907 tablet_bootstrap.cc:492] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.567052 16907 ts_tablet_manager.cc:1367] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.025s user 0.014s sys 0.009s
I20240501 08:03:38.567163 16906 tablet_bootstrap.cc:492] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.567032 16908 raft_consensus.cc:348] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.567762 16906 ts_tablet_manager.cc:1367] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.032s user 0.023s sys 0.008s
I20240501 08:03:38.567914 16908 raft_consensus.cc:374] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.568228 16908 raft_consensus.cc:729] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.568887 16908 consensus_queue.cc:260] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.569279 16907 raft_consensus.cc:348] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.569841 16907 raft_consensus.cc:374] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.570062 16907 raft_consensus.cc:729] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.570250 16906 raft_consensus.cc:348] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.570578 16907 consensus_queue.cc:260] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.570986 16906 raft_consensus.cc:374] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.571313 16906 raft_consensus.cc:729] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.571947 16906 consensus_queue.cc:260] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.572613 16907 ts_tablet_manager.cc:1398] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 08:03:38.573525 16907 tablet_bootstrap.cc:492] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.575690 16906 ts_tablet_manager.cc:1398] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20240501 08:03:38.576525 16906 tablet_bootstrap.cc:492] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.577208 16908 ts_tablet_manager.cc:1398] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240501 08:03:38.579797 16907 tablet_bootstrap.cc:654] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.580431 16908 tablet_bootstrap.cc:492] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.583700 16906 tablet_bootstrap.cc:654] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.586306 16908 tablet_bootstrap.cc:654] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.591006 16907 tablet_bootstrap.cc:492] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.591447 16907 ts_tablet_manager.cc:1367] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240501 08:03:38.593969 16907 raft_consensus.cc:348] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.594655 16907 raft_consensus.cc:374] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.594941 16907 raft_consensus.cc:729] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.595733 16907 consensus_queue.cc:260] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.598067 16907 ts_tablet_manager.cc:1398] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20240501 08:03:38.600026 16907 tablet_bootstrap.cc:492] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.601511 16906 tablet_bootstrap.cc:492] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.602005 16906 ts_tablet_manager.cc:1367] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.026s user 0.018s sys 0.004s
I20240501 08:03:38.604678 16906 raft_consensus.cc:348] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.605450 16906 raft_consensus.cc:374] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.605762 16906 raft_consensus.cc:729] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.605273 16907 tablet_bootstrap.cc:654] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.606916 16906 consensus_queue.cc:260] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.607870 16908 tablet_bootstrap.cc:492] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.608309 16908 ts_tablet_manager.cc:1367] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.028s user 0.017s sys 0.008s
I20240501 08:03:38.610826 16908 raft_consensus.cc:348] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.611533 16908 raft_consensus.cc:374] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.611822 16908 raft_consensus.cc:729] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.612466 16908 consensus_queue.cc:260] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.614470 16906 ts_tablet_manager.cc:1398] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.012s user 0.004s sys 0.002s
I20240501 08:03:38.614845 16908 ts_tablet_manager.cc:1398] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20240501 08:03:38.615402 16906 tablet_bootstrap.cc:492] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.615689 16908 tablet_bootstrap.cc:492] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.621299 16906 tablet_bootstrap.cc:654] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.621299 16908 tablet_bootstrap.cc:654] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.624025 16907 tablet_bootstrap.cc:492] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.624460 16907 ts_tablet_manager.cc:1367] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.015s
I20240501 08:03:38.627712 16907 raft_consensus.cc:348] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.628444 16907 raft_consensus.cc:374] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.628719 16907 raft_consensus.cc:729] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.630440 16908 tablet_bootstrap.cc:492] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.630928 16908 ts_tablet_manager.cc:1367] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20240501 08:03:38.631870 16907 consensus_queue.cc:260] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.632936 16911 raft_consensus.cc:482] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.633391 16911 raft_consensus.cc:504] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.634214 16907 ts_tablet_manager.cc:1398] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20240501 08:03:38.635169 16907 tablet_bootstrap.cc:492] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.633790 16908 raft_consensus.cc:348] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.635735 16911 leader_election.cc:288] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.635586 16908 raft_consensus.cc:374] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.636430 16908 raft_consensus.cc:729] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.637190 16908 consensus_queue.cc:260] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.640504 16907 tablet_bootstrap.cc:654] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.641474 16908 ts_tablet_manager.cc:1398] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20240501 08:03:38.642589 16908 tablet_bootstrap.cc:492] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.644155 16906 tablet_bootstrap.cc:492] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.644609 16906 ts_tablet_manager.cc:1367] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.029s user 0.016s sys 0.004s
I20240501 08:03:38.651959 16907 tablet_bootstrap.cc:492] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.652420 16907 ts_tablet_manager.cc:1367] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.006s
I20240501 08:03:38.653517 16906 raft_consensus.cc:348] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.654255 16906 raft_consensus.cc:374] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.654543 16906 raft_consensus.cc:729] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.654927 16908 tablet_bootstrap.cc:654] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.655166 16907 raft_consensus.cc:348] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.655848 16907 raft_consensus.cc:374] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.656105 16907 raft_consensus.cc:729] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.656648 16907 consensus_queue.cc:260] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.658365 16907 ts_tablet_manager.cc:1398] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:03:38.658526 16906 consensus_queue.cc:260] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.659639 16907 tablet_bootstrap.cc:492] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.662410 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.663166 16717 raft_consensus.cc:2454] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.664273 16829 leader_election.cc:302] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.665040 16911 raft_consensus.cc:2781] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.665429 16911 raft_consensus.cc:482] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.665776 16911 raft_consensus.cc:3037] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.667831 16907 tablet_bootstrap.cc:654] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.672099 16911 raft_consensus.cc:504] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.672621 16908 tablet_bootstrap.cc:492] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.673084 16908 ts_tablet_manager.cc:1367] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.008s
I20240501 08:03:38.675802 16906 ts_tablet_manager.cc:1398] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.031s user 0.007s sys 0.001s
I20240501 08:03:38.675642 16908 raft_consensus.cc:348] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.676299 16908 raft_consensus.cc:374] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.676563 16908 raft_consensus.cc:729] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.677201 16908 consensus_queue.cc:260] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.678192 16907 tablet_bootstrap.cc:492] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.678575 16907 ts_tablet_manager.cc:1367] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.004s
I20240501 08:03:38.679416 16908 ts_tablet_manager.cc:1398] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:03:38.680270 16908 tablet_bootstrap.cc:492] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.680833 16907 raft_consensus.cc:348] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.681480 16907 raft_consensus.cc:374] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.681738 16907 raft_consensus.cc:729] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.682380 16907 consensus_queue.cc:260] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.684438 16907 ts_tablet_manager.cc:1398] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20240501 08:03:38.685220 16907 tablet_bootstrap.cc:492] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875: Bootstrap starting.
I20240501 08:03:38.685254 16908 tablet_bootstrap.cc:654] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.686208 16906 tablet_bootstrap.cc:492] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.688302 16911 leader_election.cc:288] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.689436 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:38.689988 16717 raft_consensus.cc:3037] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.690095 16907 tablet_bootstrap.cc:654] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.695909 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.696559 16792 raft_consensus.cc:2454] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.697621 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:38.698191 16791 raft_consensus.cc:3037] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.700381 16907 tablet_bootstrap.cc:492] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875: No bootstrap required, opened a new log
I20240501 08:03:38.700429 16717 raft_consensus.cc:2454] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.700901 16907 ts_tablet_manager.cc:1367] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.011s
I20240501 08:03:38.701946 16829 leader_election.cc:302] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.703130 16907 raft_consensus.cc:348] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.703794 16907 raft_consensus.cc:374] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.704049 16907 raft_consensus.cc:729] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.704541 16791 raft_consensus.cc:2454] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.705711 16907 consensus_queue.cc:260] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.706576 16911 raft_consensus.cc:2781] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:38.707000 16908 tablet_bootstrap.cc:492] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.707427 16908 ts_tablet_manager.cc:1367] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.000s
I20240501 08:03:38.707424 16907 ts_tablet_manager.cc:1398] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:03:38.710119 16906 tablet_bootstrap.cc:654] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.709897 16908 raft_consensus.cc:348] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.710559 16908 raft_consensus.cc:374] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.710830 16908 raft_consensus.cc:729] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.711514 16908 consensus_queue.cc:260] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.712602 16911 raft_consensus.cc:686] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:38.713446 16911 consensus_queue.cc:237] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.714277 16908 ts_tablet_manager.cc:1398] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20240501 08:03:38.715255 16908 tablet_bootstrap.cc:492] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: Bootstrap starting.
I20240501 08:03:38.716440 16939 raft_consensus.cc:482] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.716912 16939 raft_consensus.cc:504] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.719513 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd455adc27f480e97b44f042a946db3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.720163 16908 tablet_bootstrap.cc:654] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.720110 16717 raft_consensus.cc:2454] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.720172 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd455adc27f480e97b44f042a946db3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.720997 16791 raft_consensus.cc:2454] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.723834 16911 raft_consensus.cc:482] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.724305 16911 raft_consensus.cc:504] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.725311 16939 leader_election.cc:288] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.726682 16911 leader_election.cc:288] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.729249 16614 catalog_manager.cc:5484] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:38.738561 16906 tablet_bootstrap.cc:492] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.738834 16913 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.739120 16906 ts_tablet_manager.cc:1367] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.053s user 0.012s sys 0.006s
I20240501 08:03:38.739480 16913 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.742028 16906 raft_consensus.cc:348] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.742801 16906 raft_consensus.cc:374] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.743188 16906 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.743897 16906 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.746292 16906 ts_tablet_manager.cc:1398] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20240501 08:03:38.747159 16906 tablet_bootstrap.cc:492] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.750936 16913 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.753274 16906 tablet_bootstrap.cc:654] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
W20240501 08:03:38.754896 16663 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:03:38.770754 16914 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.771358 16914 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.775233 16914 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.796190 16911 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.796761 16911 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.798754 16911 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.800300 16829 leader_election.cc:302] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.804157 16941 raft_consensus.cc:2781] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.804620 16941 raft_consensus.cc:482] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.817581 16941 raft_consensus.cc:3037] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.822540 16914 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.823077 16914 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.825158 16914 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.826462 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.827206 16913 raft_consensus.cc:482] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.827179 16717 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.827554 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.827850 16913 raft_consensus.cc:504] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.828361 16716 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.830226 16829 leader_election.cc:302] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.831182 16829 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.831406 16914 raft_consensus.cc:482] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.831983 16914 raft_consensus.cc:504] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.832823 16911 raft_consensus.cc:2781] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.833173 16911 raft_consensus.cc:482] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.833528 16911 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.834235 16914 leader_election.cc:288] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.837405 16939 raft_consensus.cc:2781] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.837795 16939 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.838110 16939 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.848067 16911 raft_consensus.cc:504] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.850632 16913 leader_election.cc:288] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.852205 16939 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.854154 16939 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.855376 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:38.856024 16716 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.856302 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.856941 16791 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.858217 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.858644 16908 tablet_bootstrap.cc:492] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: No bootstrap required, opened a new log
I20240501 08:03:38.858819 16791 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.860057 16908 ts_tablet_manager.cc:1367] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: Time spent bootstrapping tablet: real 0.145s user 0.056s sys 0.074s
I20240501 08:03:38.860730 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.861310 16791 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 0.
I20240501 08:03:38.862135 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.862749 16792 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 0.
I20240501 08:03:38.863615 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:38.864140 16716 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.864688 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.865222 16829 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.865553 16717 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.863914 16908 raft_consensus.cc:348] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.866400 16908 raft_consensus.cc:374] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.866695 16908 raft_consensus.cc:729] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.867313 16908 consensus_queue.cc:260] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.855412 16911 leader_election.cc:288] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.870014 16911 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.870142 16680 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.870489 16911 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.870960 16914 raft_consensus.cc:2741] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20240501 08:03:38.871799 16680 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.872501 16914 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.872871 16914 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.872984 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.865300 16792 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 0.
I20240501 08:03:38.873606 16865 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 0.
I20240501 08:03:38.873212 16914 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.873136 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.874933 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.875622 16864 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 0.
I20240501 08:03:38.875933 16908 ts_tablet_manager.cc:1398] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20240501 08:03:38.876276 16911 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.872452 16953 raft_consensus.cc:2781] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:38.874881 16906 tablet_bootstrap.cc:492] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:38.876906 16953 raft_consensus.cc:686] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:38.877068 16906 ts_tablet_manager.cc:1367] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.130s user 0.023s sys 0.044s
I20240501 08:03:38.877362 16682 leader_election.cc:302] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.877792 16953 consensus_queue.cc:237] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.872517 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.879969 16906 raft_consensus.cc:348] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.880609 16906 raft_consensus.cc:374] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:38.881150 16906 raft_consensus.cc:729] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:38.881790 16906 consensus_queue.cc:260] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.883755 16906 ts_tablet_manager.cc:1398] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:03:38.884543 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.884558 16906 tablet_bootstrap.cc:492] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:38.869328 16941 raft_consensus.cc:504] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.886636 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caf16c8ff3e4249b54bda04211f369d" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
W20240501 08:03:38.888339 16757 leader_election.cc:341] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:38.889457 16941 leader_election.cc:288] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.890697 16906 tablet_bootstrap.cc:654] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:38.891811 16717 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.893153 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:38.893844 16792 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.880616 16914 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.901228 16716 raft_consensus.cc:2379] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate fb35cf97ff15428fa584058ace6b6875 in this term.
I20240501 08:03:38.904466 16792 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.906315 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:38.906747 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.906988 16791 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.907577 16914 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 election: Requested vote from peers 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.908509 16789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd455adc27f480e97b44f042a946db3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:38.909385 16789 raft_consensus.cc:3037] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.915122 16953 raft_consensus.cc:482] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.915619 16953 raft_consensus.cc:504] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.907423 16790 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.917405 16913 raft_consensus.cc:482] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.918027 16829 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:38.918856 16941 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.919206 16941 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.919523 16941 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.917865 16913 raft_consensus.cc:504] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.922320 16953 leader_election.cc:288] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.923840 16913 leader_election.cc:288] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.924352 16913 raft_consensus.cc:482] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.927528 16941 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:38.927786 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.924772 16913 raft_consensus.cc:504] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.929957 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:38.930629 16790 raft_consensus.cc:2454] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:38.931056 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.929971 16913 leader_election.cc:288] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.931712 16864 raft_consensus.cc:2454] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 0.
I20240501 08:03:38.932524 16829 leader_election.cc:302] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:38.933131 16757 leader_election.cc:302] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:38.933336 16953 raft_consensus.cc:2781] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.933624 16941 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.933773 16953 raft_consensus.cc:482] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.934190 16953 raft_consensus.cc:3037] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.931759 16913 raft_consensus.cc:482] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.935354 16913 raft_consensus.cc:504] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.937646 16913 leader_election.cc:288] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.934782 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:38.942042 16790 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.947824 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.948736 16716 raft_consensus.cc:2379] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 947186496a094f8885f3ad358cf8d03f in current term 1: Already voted for candidate 125c40043a7044939023b53e34e3a7e7 in this term.
I20240501 08:03:38.950067 16755 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f; no voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:38.950505 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caf16c8ff3e4249b54bda04211f369d" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.951314 16913 raft_consensus.cc:2726] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:38.952646 16715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.952944 16913 raft_consensus.cc:2781] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.953416 16913 raft_consensus.cc:482] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.953348 16715 raft_consensus.cc:2454] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 0.
I20240501 08:03:38.953883 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7" is_pre_election: true
I20240501 08:03:38.954160 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.954349 16913 raft_consensus.cc:3037] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.954993 16714 raft_consensus.cc:2454] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 0.
I20240501 08:03:38.951213 16716 raft_consensus.cc:2454] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 0.
I20240501 08:03:38.960490 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:38.961160 16717 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 0.
W20240501 08:03:38.955261 16757 leader_election.cc:341] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:38.964231 16755 leader_election.cc:302] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters: 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:38.965502 16755 leader_election.cc:302] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters: 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:38.968837 16791 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.969508 16953 raft_consensus.cc:504] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.908643 16939 raft_consensus.cc:482] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:38.917091 16789 raft_consensus.cc:2454] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:38.908670 16914 raft_consensus.cc:2781] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.974829 16913 raft_consensus.cc:504] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.972565 16939 raft_consensus.cc:504] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.978181 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:38.978328 16866 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 0.
I20240501 08:03:38.972807 16914 raft_consensus.cc:482] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.973186 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:38.980943 16864 raft_consensus.cc:2379] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 947186496a094f8885f3ad358cf8d03f in current term 1: Already voted for candidate 125c40043a7044939023b53e34e3a7e7 in this term.
I20240501 08:03:38.980743 16755 leader_election.cc:302] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.980158 16914 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.976655 16953 leader_election.cc:288] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.982270 16939 leader_election.cc:288] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:38.976734 16913 leader_election.cc:288] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.985076 16959 raft_consensus.cc:2781] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20240501 08:03:38.973950 16682 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.131:34079, user_credentials={real_user=slave}} blocked reactor thread for 96837.3us
I20240501 08:03:38.989017 16913 raft_consensus.cc:2781] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.989439 16913 raft_consensus.cc:482] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.989785 16913 raft_consensus.cc:3037] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.990620 16958 raft_consensus.cc:2781] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:38.991108 16958 raft_consensus.cc:482] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:38.991484 16958 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:38.996104 16913 raft_consensus.cc:504] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:38.994915 16614 catalog_manager.cc:5484] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:38.997356 16829 leader_election.cc:302] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:38.998031 16913 leader_election.cc:288] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:38.999089 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caf16c8ff3e4249b54bda04211f369d" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7"
W20240501 08:03:39.000161 16757 leader_election.cc:341] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:39.000382 16906 tablet_bootstrap.cc:492] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:39.000887 16906 ts_tablet_manager.cc:1367] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.116s user 0.010s sys 0.023s
I20240501 08:03:39.001600 16790 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:39.002372 16939 raft_consensus.cc:2781] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.003491 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.003350 16906 raft_consensus.cc:348] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.004204 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.004982 16866 raft_consensus.cc:2379] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fb35cf97ff15428fa584058ace6b6875 in current term 1: Already voted for candidate 125c40043a7044939023b53e34e3a7e7 in this term.
I20240501 08:03:39.005213 16714 raft_consensus.cc:2379] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate fb35cf97ff15428fa584058ace6b6875 in this term.
I20240501 08:03:39.005560 16715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.004812 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:39.005903 16906 raft_consensus.cc:374] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:39.006152 16715 raft_consensus.cc:3037] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.006342 16906 raft_consensus.cc:729] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:39.005645 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd455adc27f480e97b44f042a946db3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.006170 16939 raft_consensus.cc:686] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.007107 16717 raft_consensus.cc:3037] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.007572 16939 consensus_queue.cc:237] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.007308 16906 consensus_queue.cc:260] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.010702 16906 ts_tablet_manager.cc:1398] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20240501 08:03:39.011569 16906 tablet_bootstrap.cc:492] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7: Bootstrap starting.
I20240501 08:03:39.013057 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:39.011042 16959 raft_consensus.cc:482] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:39.015588 16715 raft_consensus.cc:2454] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:39.015689 16959 raft_consensus.cc:3037] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.015949 16682 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb35cf97ff15428fa584058ace6b6875; no voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f
I20240501 08:03:39.017938 16914 raft_consensus.cc:504] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.020864 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.014689 16716 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 0.
I20240501 08:03:39.022347 16790 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.025228 16955 raft_consensus.cc:2726] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:03:39.024842 16958 raft_consensus.cc:504] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.026958 16829 leader_election.cc:302] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.027762 16953 raft_consensus.cc:2781] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:03:39.028093 16953 raft_consensus.cc:482] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:03:39.028403 16953 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.028883 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.029590 16958 leader_election.cc:288] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
W20240501 08:03:39.030689 16755 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.129:42471, user_credentials={real_user=slave}} blocked reactor thread for 50056.4us
I20240501 08:03:39.031489 16829 leader_election.cc:302] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:39.033694 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caf16c8ff3e4249b54bda04211f369d" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.035465 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.035619 16789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:39.036103 16714 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.036363 16789 raft_consensus.cc:2379] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate 947186496a094f8885f3ad358cf8d03f in this term.
I20240501 08:03:39.034559 16716 raft_consensus.cc:3037] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.033152 16941 raft_consensus.cc:2781] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.032069 16959 raft_consensus.cc:504] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.034919 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.042855 16792 raft_consensus.cc:2379] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate 947186496a094f8885f3ad358cf8d03f in this term.
I20240501 08:03:39.038651 16941 raft_consensus.cc:686] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.042691 16614 catalog_manager.cc:5484] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:39.044039 16941 consensus_queue.cc:237] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.046291 16790 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 1.
I20240501 08:03:39.039887 16717 raft_consensus.cc:2454] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 1.
I20240501 08:03:39.042088 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.033313 16914 leader_election.cc:288] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.047588 16864 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.035071 16829 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters: fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.040284 16715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.049551 16680 leader_election.cc:302] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.050056 16829 leader_election.cc:302] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.053521 16911 raft_consensus.cc:2781] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.054152 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.054548 16911 raft_consensus.cc:686] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.055423 16911 consensus_queue.cc:237] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.057106 16864 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 1.
W20240501 08:03:39.055536 16757 leader_election.cc:341] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:03:39.051434 16716 raft_consensus.cc:2454] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 1.
I20240501 08:03:39.052032 16959 leader_election.cc:288] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.052484 16914 raft_consensus.cc:2781] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.052876 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.052588 16953 raft_consensus.cc:504] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.059682 16714 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 1.
I20240501 08:03:39.053092 16715 raft_consensus.cc:2379] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 947186496a094f8885f3ad358cf8d03f in current term 1: Already voted for candidate 125c40043a7044939023b53e34e3a7e7 in this term.
I20240501 08:03:39.062592 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.062381 16755 leader_election.cc:302] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters: 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:39.064095 16906 tablet_bootstrap.cc:654] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7: Neither blocks nor log segments found. Creating new log.
I20240501 08:03:39.053387 16939 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.064441 16959 raft_consensus.cc:2781] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.064625 16939 raft_consensus.cc:686] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.064882 16959 raft_consensus.cc:686] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:39.059659 16717 raft_consensus.cc:3037] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.061488 16914 raft_consensus.cc:686] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.061877 16953 leader_election.cc:288] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:39.062132 16866 raft_consensus.cc:2379] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 947186496a094f8885f3ad358cf8d03f in current term 1: Already voted for candidate 125c40043a7044939023b53e34e3a7e7 in this term.
I20240501 08:03:39.066885 16959 consensus_queue.cc:237] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.065460 16939 consensus_queue.cc:237] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:39.066790 16914 consensus_queue.cc:237] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.069959 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.070705 16790 raft_consensus.cc:2379] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate 947186496a094f8885f3ad358cf8d03f in this term.
I20240501 08:03:39.071504 16755 leader_election.cc:302] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f; no voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.073194 16960 raft_consensus.cc:2726] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:03:39.074522 16755 leader_election.cc:302] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters: 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:39.075379 16829 leader_election.cc:302] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7; no voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.076122 16953 raft_consensus.cc:2726] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:03:39.076408 16960 raft_consensus.cc:2781] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.077041 16960 raft_consensus.cc:686] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:39.077827 16960 consensus_queue.cc:237] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.079682 16614 catalog_manager.cc:5484] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:39.081434 16614 catalog_manager.cc:5484] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:39.084302 16717 raft_consensus.cc:2454] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 1.
I20240501 08:03:39.087159 16755 leader_election.cc:302] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters: 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:39.088073 16958 raft_consensus.cc:2781] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:03:39.088529 16958 raft_consensus.cc:686] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:39.089337 16958 consensus_queue.cc:237] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.094623 16911 raft_consensus.cc:482] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:39.095082 16911 raft_consensus.cc:504] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.095794 16906 tablet_bootstrap.cc:492] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7: No bootstrap required, opened a new log
I20240501 08:03:39.096228 16906 ts_tablet_manager.cc:1367] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7: Time spent bootstrapping tablet: real 0.085s user 0.027s sys 0.003s
I20240501 08:03:39.098824 16911 leader_election.cc:288] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:39.098564 16906 raft_consensus.cc:348] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.099301 16906 raft_consensus.cc:374] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:03:39.099573 16906 raft_consensus.cc:729] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Initialized, Role: FOLLOWER
I20240501 08:03:39.099692 16615 catalog_manager.cc:5484] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 0 to 1, leader changed from <none> to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 1 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.106763 16906 consensus_queue.cc:260] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.108803 16906 ts_tablet_manager.cc:1398] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20240501 08:03:39.110221 16614 catalog_manager.cc:5484] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.110801 16613 catalog_manager.cc:5484] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.111699 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caf16c8ff3e4249b54bda04211f369d" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:39.112638 16717 raft_consensus.cc:2379] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate 947186496a094f8885f3ad358cf8d03f in this term.
I20240501 08:03:39.114544 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9caf16c8ff3e4249b54bda04211f369d" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:39.115602 16829 leader_election.cc:302] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7; no voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.116648 16911 raft_consensus.cc:3037] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.123629 16613 catalog_manager.cc:5484] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 0 to 1, leader changed from <none> to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 1 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.125291 16911 raft_consensus.cc:2726] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:39.125406 16613 catalog_manager.cc:5484] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 0 to 1, leader changed from <none> to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 1 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.157495 16911 consensus_queue.cc:1035] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.172503 16911 consensus_queue.cc:1035] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.179996 16866 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:39.180723 16866 raft_consensus.cc:595] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.185509 16866 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:39.186008 16866 raft_consensus.cc:595] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.190375 16866 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:39.190815 16866 raft_consensus.cc:595] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.194666 16866 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "afd455adc27f480e97b44f042a946db3"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:39.195166 16866 raft_consensus.cc:595] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.198930 16866 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:39.199345 16866 raft_consensus.cc:595] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.205303 16866 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:39.205803 16866 raft_consensus.cc:595] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Received request to transfer leadership to TS 947186496a094f8885f3ad358cf8d03f
I20240501 08:03:39.215842 16717 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:32980
I20240501 08:03:39.216307 16717 raft_consensus.cc:595] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 LEADER]: Received request to transfer leadership to TS 947186496a094f8885f3ad358cf8d03f
I20240501 08:03:39.228654 16790 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
mode: GRACEFUL
new_leader_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:52654
I20240501 08:03:39.229133 16790 raft_consensus.cc:595] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Received request to transfer leadership to TS 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:39.388007 16911 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.400305 16953 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.406695 16939 raft_consensus.cc:979] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.407186 16953 raft_consensus.cc:1067] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.408537 16717 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:39.408974 16717 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.409251 16717 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.413841 16717 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.415236 16717 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.416098 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.416132 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.416839 16866 raft_consensus.cc:3032] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.416888 16790 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.417402 16866 raft_consensus.cc:729] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.418169 16866 consensus_queue.cc:260] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.419399 16866 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.421952 16790 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.422750 16680 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.423380 16914 raft_consensus.cc:2781] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.423840 16914 raft_consensus.cc:686] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.424814 16866 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.424614 16914 consensus_queue.cc:237] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.431238 16614 catalog_manager.cc:5484] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 1 to 2, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.447947 16953 consensus_queue.cc:1035] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.459728 16911 consensus_queue.cc:1035] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.459898 16958 consensus_queue.cc:1035] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.459667 16941 consensus_queue.cc:1035] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:03:39.469964 16939 raft_consensus.cc:979] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.470396 16939 raft_consensus.cc:1067] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.476389 16714 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "afd455adc27f480e97b44f042a946db3"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:39.476966 16714 raft_consensus.cc:482] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.477388 16941 raft_consensus.cc:979] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.477813 16941 raft_consensus.cc:1067] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.477401 16714 raft_consensus.cc:3037] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.476858 16939 consensus_queue.cc:1035] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20240501 08:03:39.482295 16939 raft_consensus.cc:482] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:03:39.482775 16939 raft_consensus.cc:504] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.484756 16939 leader_election.cc:288] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:39.483287 16913 consensus_queue.cc:1035] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240501 08:03:39.485396 16714 raft_consensus.cc:504] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.487232 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947186496a094f8885f3ad358cf8d03f" is_pre_election: true
I20240501 08:03:39.488866 16714 leader_election.cc:288] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.489760 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd455adc27f480e97b44f042a946db3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.492761 16914 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.493170 16714 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:39.493736 16714 raft_consensus.cc:482] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.493960 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb35cf97ff15428fa584058ace6b6875" is_pre_election: true
I20240501 08:03:39.494092 16714 raft_consensus.cc:3037] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.495044 16716 raft_consensus.cc:2379] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 125c40043a7044939023b53e34e3a7e7 in current term 1: Already voted for candidate 947186496a094f8885f3ad358cf8d03f in this term.
I20240501 08:03:39.494087 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd455adc27f480e97b44f042a946db3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.495935 16864 raft_consensus.cc:3032] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.496001 16941 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.496739 16913 consensus_queue.cc:1035] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.496338 16864 raft_consensus.cc:729] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.497776 16864 consensus_queue.cc:260] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.498376 16829 leader_election.cc:302] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7; no voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:39.498745 16864 raft_consensus.cc:3037] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.499547 16939 raft_consensus.cc:3037] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:03:39.499742 16941 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.500609 16714 raft_consensus.cc:504] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.503593 16714 leader_election.cc:288] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.506021 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.506260 16939 raft_consensus.cc:2726] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:03:39.506641 16865 raft_consensus.cc:3032] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.506992 16865 raft_consensus.cc:729] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.507311 16790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364c7a4942e24357bc5446f5d6ef7b01" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.507894 16790 raft_consensus.cc:3037] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.507730 16865 consensus_queue.cc:260] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.509111 16865 raft_consensus.cc:3037] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.513937 16717 raft_consensus.cc:1226] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Rejecting Update request from peer 125c40043a7044939023b53e34e3a7e7 for earlier term 1. Current term is 2. Ops: []
I20240501 08:03:39.513106 16790 raft_consensus.cc:2454] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.520886 16680 leader_election.cc:302] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.521729 16914 raft_consensus.cc:2781] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.522136 16914 raft_consensus.cc:686] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.522781 16914 consensus_queue.cc:237] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.526060 16864 raft_consensus.cc:2454] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.530900 16682 leader_election.cc:302] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters: 947186496a094f8885f3ad358cf8d03f
I20240501 08:03:39.532485 16914 raft_consensus.cc:2781] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.532912 16914 raft_consensus.cc:686] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.533694 16914 consensus_queue.cc:237] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.538043 16913 consensus_queue.cc:1035] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:03:39.540097 16953 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.540827 16914 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.556034 16865 raft_consensus.cc:2454] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.559633 16914 raft_consensus.cc:979] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:39.560060 16914 raft_consensus.cc:1067] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 LEADER]: Signalling peer 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:39.563984 16939 raft_consensus.cc:979] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:39.564452 16939 raft_consensus.cc:1067] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Signalling peer 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:39.566793 16792 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:52636
I20240501 08:03:39.567138 16960 consensus_queue.cc:1035] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.567339 16792 raft_consensus.cc:482] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.570154 16792 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.569231 16791 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:52640
I20240501 08:03:39.571449 16791 raft_consensus.cc:482] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.571852 16791 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.573040 16615 catalog_manager.cc:5484] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 1 to 2, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.578579 16911 raft_consensus.cc:979] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.579037 16939 raft_consensus.cc:1067] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.588675 16791 raft_consensus.cc:504] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.586598 16792 raft_consensus.cc:504] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.591300 16791 leader_election.cc:288] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 2 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.596890 16789 raft_consensus.cc:1226] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Rejecting Update request from peer 125c40043a7044939023b53e34e3a7e7 for earlier term 1. Current term is 2. Ops: []
I20240501 08:03:39.600433 16960 consensus_queue.cc:1035] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.601852 16959 raft_consensus.cc:979] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: : Instructing follower 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:39.602262 16959 raft_consensus.cc:1067] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Signalling peer 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:39.603657 16865 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:48118
I20240501 08:03:39.603758 16939 consensus_queue.cc:1046] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.604413 16865 raft_consensus.cc:482] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.604496 16941 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:03:39.594270 16714 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:39.593675 16866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.604740 16939 raft_consensus.cc:3032] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.594213 16715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.605444 16714 raft_consensus.cc:482] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.604871 16865 raft_consensus.cc:3037] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.605466 16939 raft_consensus.cc:729] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.605883 16715 raft_consensus.cc:3032] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.606616 16715 raft_consensus.cc:729] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.606804 16939 consensus_queue.cc:260] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.607369 16715 consensus_queue.cc:260] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.607924 16939 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.608304 16866 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.608598 16715 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.605967 16714 raft_consensus.cc:3037] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.613780 16717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.613945 16792 leader_election.cc:288] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 2 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.614027 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.614389 16717 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.618135 16714 raft_consensus.cc:504] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.618957 16717 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 2.
I20240501 08:03:39.619452 16865 raft_consensus.cc:504] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.620761 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.621399 16792 raft_consensus.cc:3037] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.621930 16866 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 2.
I20240501 08:03:39.623165 16715 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 2.
I20240501 08:03:39.624186 16755 leader_election.cc:302] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.625072 16755 leader_election.cc:302] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.625375 16960 raft_consensus.cc:2781] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.625881 16960 raft_consensus.cc:686] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:39.626057 16865 leader_election.cc:288] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 2 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:39.626161 16959 raft_consensus.cc:2781] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.627972 16959 raft_consensus.cc:686] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:39.628865 16792 raft_consensus.cc:2454] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.628756 16959 consensus_queue.cc:237] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.626683 16960 consensus_queue.cc:237] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.629987 16680 leader_election.cc:302] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.630771 16989 raft_consensus.cc:2781] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.631176 16989 raft_consensus.cc:686] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.631935 16989 consensus_queue.cc:237] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.620043 16714 leader_election.cc:288] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:39.638046 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.637692 16614 catalog_manager.cc:5484] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 1 to 2, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.638655 16791 raft_consensus.cc:3032] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.638990 16791 raft_consensus.cc:729] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:39.639698 16791 consensus_queue.cc:260] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.640861 16791 raft_consensus.cc:3037] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.642164 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d937a5aa00c74349a0953014727fdf2f" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.642861 16865 raft_consensus.cc:3032] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.643182 16865 raft_consensus.cc:729] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.643734 16864 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 2.
I20240501 08:03:39.643970 16865 consensus_queue.cc:260] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.645717 16865 raft_consensus.cc:3037] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.647611 16791 raft_consensus.cc:2454] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 2.
I20240501 08:03:39.648535 16829 leader_election.cc:302] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:39.649375 16939 raft_consensus.cc:2781] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.649818 16939 raft_consensus.cc:686] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 2 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.650566 16939 consensus_queue.cc:237] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:39.653586 16715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "478b286082d34eecac6cc50fdcbe5e19" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:39.654290 16715 raft_consensus.cc:3037] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.657126 16613 catalog_manager.cc:5484] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 1 to 2, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 2 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.658074 16865 raft_consensus.cc:2454] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.665313 16615 catalog_manager.cc:5484] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 1 to 2, leader changed from 947186496a094f8885f3ad358cf8d03f (127.15.168.130) to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 2 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:39.666405 16939 raft_consensus.cc:979] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.666669 16715 raft_consensus.cc:2454] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 2.
I20240501 08:03:39.666823 16939 raft_consensus.cc:1067] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:39.669731 16715 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:39.670260 16715 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:39.670630 16715 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.676251 16614 catalog_manager.cc:5484] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 1 to 2, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 2 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.676592 16715 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:39.678383 16715 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Requested vote from peers 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:39.678843 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:39.679131 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:39.679394 16865 raft_consensus.cc:3032] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:39.679534 16791 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.679673 16865 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:39.680233 16865 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:39.681015 16865 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.684255 16791 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.685194 16680 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:39.686002 16989 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:39.686095 16865 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:39.686444 16989 raft_consensus.cc:686] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:39.687158 16989 consensus_queue.cc:237] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:39.689164 16613 catalog_manager.cc:5484] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 1 to 2, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.700882 16613 catalog_manager.cc:5484] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 1 to 2, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:39.931383 16865 raft_consensus.cc:1261] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:39.932463 16989 consensus_queue.cc:1035] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:39.940254 16791 raft_consensus.cc:3037] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:39.943388 16865 raft_consensus.cc:1261] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:39.944645 16989 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:39.947252 16791 raft_consensus.cc:1261] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:39.948613 16989 consensus_queue.cc:1035] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:39.953169 16791 raft_consensus.cc:1261] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:39.954587 16989 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:39.986433 16792 raft_consensus.cc:1261] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:39.987505 16914 consensus_queue.cc:1035] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:39.993965 16865 raft_consensus.cc:1261] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:39.995358 16988 consensus_queue.cc:1035] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.033497 16715 raft_consensus.cc:1261] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.034888 16953 consensus_queue.cc:1035] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:03:40.046265 16715 raft_consensus.cc:1261] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.047827 16960 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.046300 16792 raft_consensus.cc:1261] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.052801 16953 consensus_queue.cc:1035] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:03:40.055074 16865 raft_consensus.cc:1261] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.061656 16716 raft_consensus.cc:1261] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.062281 16988 consensus_queue.cc:1035] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.066602 16865 raft_consensus.cc:1261] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.067442 16960 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
W20240501 08:03:40.128656 16682 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.131:34079, user_credentials={real_user=slave}} blocked reactor thread for 70246.1us
I20240501 08:03:40.221568 16864 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.223165 16988 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.294739 16865 raft_consensus.cc:1261] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.295846 16959 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.297710 16792 raft_consensus.cc:1261] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.299499 16792 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:40.300495 16959 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.307971 16989 consensus_queue.cc:1035] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:40.309173 16989 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240501 08:03:41.126170 16646 debug-util.cc:398] Leaking SignalData structure 0x7b0800051c80 after lost signal to thread 16037
W20240501 08:03:41.126871 16646 debug-util.cc:398] Leaking SignalData structure 0x7b080004f3a0 after lost signal to thread 16649
W20240501 08:03:41.127600 16646 debug-util.cc:398] Leaking SignalData structure 0x7b0800194ac0 after lost signal to thread 16741
W20240501 08:03:41.128268 16646 debug-util.cc:398] Leaking SignalData structure 0x7b08000f2dc0 after lost signal to thread 16816
W20240501 08:03:41.128906 16646 debug-util.cc:398] Leaking SignalData structure 0x7b0800060f80 after lost signal to thread 16890
I20240501 08:03:42.289191 16034 auto_leader_rebalancer-test.cc:331] The leader distribution is
I20240501 08:03:42.290045 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:42.310156 16716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:32990
I20240501 08:03:42.310775 16716 raft_consensus.cc:595] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Received request to transfer leadership to TS 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:42.313202 16716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:32990
I20240501 08:03:42.313802 16716 raft_consensus.cc:595] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Received request to transfer leadership to TS 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:42.314714 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240501 08:03:42.315119 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:42.335393 17001 raft_consensus.cc:979] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:42.335824 17001 raft_consensus.cc:1067] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Signalling peer 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:42.337214 16864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:48124
I20240501 08:03:42.337767 16864 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:42.338091 16864 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:42.342669 16864 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:42.344223 16864 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 3 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:42.345022 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:42.345460 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:42.345661 16716 raft_consensus.cc:3032] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:42.346015 16791 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:42.350543 16791 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 3.
I20240501 08:03:42.346010 16716 raft_consensus.cc:729] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:42.351471 16829 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:42.351393 16716 consensus_queue.cc:260] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:42.352182 16953 raft_consensus.cc:2781] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:42.352757 16716 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:42.353193 16953 raft_consensus.cc:686] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:42.354017 16953 consensus_queue.cc:237] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:42.359623 16716 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 3.
I20240501 08:03:42.361449 16614 catalog_manager.cc:5484] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 2 to 3, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 3 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:42.637903 17001 raft_consensus.cc:979] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:42.638355 17000 raft_consensus.cc:1067] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Signalling peer 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:42.639669 16864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:48124
I20240501 08:03:42.640153 16864 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:42.640432 16864 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:42.645606 16864 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:42.647064 16864 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 3 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:42.648065 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:42.648509 16791 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:42.648739 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:42.649320 16716 raft_consensus.cc:3032] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:42.649704 16716 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:42.650408 16716 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:42.651582 16716 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:42.652984 16791 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 3.
I20240501 08:03:42.653867 16829 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:42.654624 16953 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:42.654982 16953 raft_consensus.cc:686] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:42.655740 16953 consensus_queue.cc:237] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:42.660650 16716 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 3.
I20240501 08:03:42.662798 16614 catalog_manager.cc:5484] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 2 to 3, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 3 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:42.864557 16716 raft_consensus.cc:1261] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:42.865800 16953 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:42.880128 16791 raft_consensus.cc:1261] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:42.881624 17004 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:43.108000 16791 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:43.109118 16953 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:43.119194 16716 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:43.120622 16953 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:43.315855 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:43.321835 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:43.322408 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:44.323176 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:44.328307 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:44.328652 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:45.329581 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:45.335042 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:45.335448 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:46.336198 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:46.342236 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:46.342623 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:47.343322 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:47.348419 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:47.348906 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:48.543743 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:48.548800 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:48.549171 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
W20240501 08:03:49.284511 16738 debug-util.cc:398] Leaking SignalData structure 0x7b080010f620 after lost signal to thread 16037
W20240501 08:03:49.285264 16738 debug-util.cc:398] Leaking SignalData structure 0x7b08001b5ce0 after lost signal to thread 16649
W20240501 08:03:49.286041 16738 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1700 after lost signal to thread 16741
W20240501 08:03:49.286692 16738 debug-util.cc:398] Leaking SignalData structure 0x7b0800110320 after lost signal to thread 16816
W20240501 08:03:49.287334 16738 debug-util.cc:398] Leaking SignalData structure 0x7b080004fc20 after lost signal to thread 16890
I20240501 08:03:49.549912 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:49.555619 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:49.555979 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:50.556690 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:50.563095 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:50.563910 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:51.564694 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:51.570724 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:51.571107 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:52.594491 16034 auto_leader_rebalancer-test.cc:350] The leader distribution is
I20240501 08:03:52.598410 16864 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:52.599004 16864 raft_consensus.cc:595] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:52.605335 16864 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:48130
I20240501 08:03:52.605737 16864 raft_consensus.cc:595] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:52.613718 16791 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "321032c3a694400e820755e4cadd5815"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:52654
I20240501 08:03:52.614125 16791 raft_consensus.cc:595] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:52.618495 16791 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:52654
I20240501 08:03:52.618970 16791 raft_consensus.cc:595] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:52.622782 16791 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
mode: GRACEFUL
new_leader_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:52654
I20240501 08:03:52.623164 16791 raft_consensus.cc:595] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Received request to transfer leadership to TS fb35cf97ff15428fa584058ace6b6875
I20240501 08:03:52.686786 17033 raft_consensus.cc:979] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.687249 17031 raft_consensus.cc:1067] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.688515 16715 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32964
I20240501 08:03:52.688918 16715 raft_consensus.cc:482] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:52.689225 16715 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:52.694129 16715 raft_consensus.cc:504] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.695523 16715 leader_election.cc:288] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 3 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:52.696631 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:52.696559 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:52.697343 16791 raft_consensus.cc:3032] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:52.697731 16791 raft_consensus.cc:729] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:52.697427 16864 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:52.698812 16791 consensus_queue.cc:260] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.700038 16791 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:52.701148 17034 raft_consensus.cc:979] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.701638 17030 raft_consensus.cc:1067] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.703193 16715 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:52.703559 16715 raft_consensus.cc:482] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:52.703817 16715 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:52.704041 16864 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 3.
I20240501 08:03:52.705200 16682 leader_election.cc:302] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:52.706051 17032 raft_consensus.cc:2781] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:52.706483 16791 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 3.
I20240501 08:03:52.707105 17032 raft_consensus.cc:686] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:52.707953 17032 consensus_queue.cc:237] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.708809 16715 raft_consensus.cc:504] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.710484 16715 leader_election.cc:288] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 4 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:52.710953 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:52.711321 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f67ff0cac9ea438196146004f9f9d740" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:52.711522 16791 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:52.711913 16864 raft_consensus.cc:3032] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:52.712224 16864 raft_consensus.cc:729] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:52.712997 16864 consensus_queue.cc:260] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.714166 16864 raft_consensus.cc:3037] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:52.717694 16791 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 4.
I20240501 08:03:52.718712 16680 leader_election.cc:302] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:52.719311 17032 raft_consensus.cc:2781] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:52.719414 16864 raft_consensus.cc:2454] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 4.
I20240501 08:03:52.719722 17032 raft_consensus.cc:686] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:52.720561 17032 consensus_queue.cc:237] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.720813 16612 catalog_manager.cc:5484] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 2 to 3, leader changed from 947186496a094f8885f3ad358cf8d03f (127.15.168.130) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 3 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:52.738143 16612 catalog_manager.cc:5484] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 3 to 4, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 4 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:52.832823 17033 raft_consensus.cc:979] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.833195 17031 raft_consensus.cc:1067] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.834410 16715 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "321032c3a694400e820755e4cadd5815"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32964
I20240501 08:03:52.834825 16715 raft_consensus.cc:482] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:52.835072 16715 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:52.839170 16715 raft_consensus.cc:504] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.840399 16715 leader_election.cc:288] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:52.841117 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:52.841229 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:52.841656 16791 raft_consensus.cc:3032] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:03:52.841791 16864 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:52.841931 16791 raft_consensus.cc:729] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:52.842571 16791 consensus_queue.cc:260] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.843397 16791 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:03:52.845664 16864 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:52.846390 16682 leader_election.cc:302] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:52.847421 16791 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 2.
I20240501 08:03:52.847025 17032 raft_consensus.cc:2781] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:03:52.848012 17032 raft_consensus.cc:686] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:52.848817 17032 consensus_queue.cc:237] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:52.854647 16612 catalog_manager.cc:5484] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 1 to 2, leader changed from 947186496a094f8885f3ad358cf8d03f (127.15.168.130) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:52.987586 17034 raft_consensus.cc:979] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.987991 17030 raft_consensus.cc:1067] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:52.989641 16716 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32952
I20240501 08:03:52.990046 16716 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:52.990278 16716 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:52.994573 16716 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:52.995877 16716 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 4 election: Requested vote from peers 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:52.996464 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:52.996985 16864 raft_consensus.cc:3032] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:52.997057 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:52.997205 16864 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:52.997560 16791 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:52.997861 16864 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:52.998679 16864 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:53.001367 16791 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 4.
I20240501 08:03:53.002449 16864 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 4.
I20240501 08:03:53.002185 16680 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:53.003293 17032 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:53.003603 17032 raft_consensus.cc:686] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:53.004158 17032 consensus_queue.cc:237] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:53.009835 16612 catalog_manager.cc:5484] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 3 to 4, leader changed from 125c40043a7044939023b53e34e3a7e7 (127.15.168.131) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 4 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:53.148001 17033 raft_consensus.cc:979] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: : Instructing follower fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:53.148365 17031 raft_consensus.cc:1067] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Signalling peer fb35cf97ff15428fa584058ace6b6875 to start an election
I20240501 08:03:53.149668 16716 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
from {username='slave'} at 127.0.0.1:32964
I20240501 08:03:53.150064 16716 raft_consensus.cc:482] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:53.150322 16716 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:53.155328 16716 raft_consensus.cc:504] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:53.156659 16716 leader_election.cc:288] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 3 election: Requested vote from peers 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:53.157542 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:53.157563 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "fb35cf97ff15428fa584058ace6b6875" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:53.158233 16864 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:53.158206 16791 raft_consensus.cc:3032] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:53.158790 16791 raft_consensus.cc:729] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:53.159451 16791 consensus_queue.cc:260] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:53.160528 16791 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:53.163997 16864 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 3.
I20240501 08:03:53.164875 16682 leader_election.cc:302] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:53.165570 17032 raft_consensus.cc:2781] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:53.165916 17032 raft_consensus.cc:686] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Becoming Leader. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:53.166000 16791 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb35cf97ff15428fa584058ace6b6875 in term 3.
I20240501 08:03:53.166710 17032 consensus_queue.cc:237] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:53.172086 16791 raft_consensus.cc:1261] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:53.173069 17032 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:03:53.174296 16612 catalog_manager.cc:5484] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 reported cstate change: term changed from 2 to 3, leader changed from 947186496a094f8885f3ad358cf8d03f (127.15.168.130) to fb35cf97ff15428fa584058ace6b6875 (127.15.168.129). New cstate: current_term: 3 leader_uuid: "fb35cf97ff15428fa584058ace6b6875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:53.182029 16864 raft_consensus.cc:1261] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:53.183416 17032 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:53.299114 16864 raft_consensus.cc:1261] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:53.300184 17035 consensus_queue.cc:1035] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:03:53.308244 16791 raft_consensus.cc:1261] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240501 08:03:53.309537 17032 consensus_queue.cc:1035] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:03:53.319453 16791 raft_consensus.cc:1261] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:53.320509 17032 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:53.329205 16864 raft_consensus.cc:1261] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:53.330576 17051 consensus_queue.cc:1035] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240501 08:03:53.409219 16864 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:53.410493 17051 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:53.419559 16791 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:53.420655 17035 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:53.587877 16791 raft_consensus.cc:1261] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:53.588919 17035 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:53.596592 16864 raft_consensus.cc:1261] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Refusing update from remote peer fb35cf97ff15428fa584058ace6b6875: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:53.599280 17035 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:55.649677 16034 auto_leader_rebalancer-test.cc:365] The leader distribution is
I20240501 08:03:55.650367 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:55.657207 16716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "321032c3a694400e820755e4cadd5815"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:32990
I20240501 08:03:55.657822 16716 raft_consensus.cc:595] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Received request to transfer leadership to TS 947186496a094f8885f3ad358cf8d03f
I20240501 08:03:55.659924 16716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:32990
I20240501 08:03:55.660341 16716 raft_consensus.cc:595] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Received request to transfer leadership to TS 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:55.662101 16716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:32990
I20240501 08:03:55.662487 16716 raft_consensus.cc:595] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Received request to transfer leadership to TS 947186496a094f8885f3ad358cf8d03f
I20240501 08:03:55.664223 16716 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
mode: GRACEFUL
new_leader_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:32990
I20240501 08:03:55.664608 16716 raft_consensus.cc:595] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Received request to transfer leadership to TS 125c40043a7044939023b53e34e3a7e7
I20240501 08:03:55.665478 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 4
I20240501 08:03:55.665863 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:55.694867 17061 raft_consensus.cc:979] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:55.695410 17060 raft_consensus.cc:1067] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Signalling peer 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:55.696684 16865 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:48124
I20240501 08:03:55.697122 16865 raft_consensus.cc:482] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:55.697477 16865 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:55.702872 16865 raft_consensus.cc:504] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:55.704758 16865 leader_election.cc:288] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 4 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:55.705559 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:55.705899 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44feef2d33b349bdb3cd2ae0389a5745" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:55.706190 16716 raft_consensus.cc:3032] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:55.706352 16791 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:55.706521 16716 raft_consensus.cc:729] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:55.707230 16716 consensus_queue.cc:260] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:55.708242 16716 raft_consensus.cc:3037] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:55.711064 16791 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 4.
I20240501 08:03:55.712087 16829 leader_election.cc:302] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:55.712917 17030 raft_consensus.cc:2781] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:55.713471 16716 raft_consensus.cc:2454] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 4.
I20240501 08:03:55.713837 17030 raft_consensus.cc:686] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 4 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:55.714779 17030 consensus_queue.cc:237] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:55.720757 16614 catalog_manager.cc:5484] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 3 to 4, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 4 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } }
I20240501 08:03:56.381318 16716 raft_consensus.cc:1261] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:56.382426 17062 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:56.416121 17069 raft_consensus.cc:979] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:56.417161 17061 raft_consensus.cc:979] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:56.417670 17061 raft_consensus.cc:1067] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Signalling peer 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:56.417122 17074 raft_consensus.cc:1067] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Signalling peer 947186496a094f8885f3ad358cf8d03f to start an election
I20240501 08:03:56.419507 17060 raft_consensus.cc:979] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: : Instructing follower 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:56.419932 17060 raft_consensus.cc:1067] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Signalling peer 125c40043a7044939023b53e34e3a7e7 to start an election
I20240501 08:03:56.427130 16864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3"
dest_uuid: "125c40043a7044939023b53e34e3a7e7"
from {username='slave'} at 127.0.0.1:48124
I20240501 08:03:56.427793 16864 raft_consensus.cc:482] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:56.428213 16864 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Advancing to term 5
I20240501 08:03:56.435009 16864 raft_consensus.cc:504] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:56.441210 16864 leader_election.cc:288] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 5 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 947186496a094f8885f3ad358cf8d03f (127.15.168.130:40167)
I20240501 08:03:56.458263 16792 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "321032c3a694400e820755e4cadd5815"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:52640
I20240501 08:03:56.458972 16789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "947186496a094f8885f3ad358cf8d03f"
I20240501 08:03:56.459578 16789 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Advancing to term 5
I20240501 08:03:56.459677 16790 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86"
dest_uuid: "947186496a094f8885f3ad358cf8d03f"
from {username='slave'} at 127.0.0.1:52640
I20240501 08:03:56.460294 16790 raft_consensus.cc:482] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:56.460618 16790 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:56.467589 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd7076d607314b098f3f1152781ae2f3" candidate_uuid: "125c40043a7044939023b53e34e3a7e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:56.468127 16792 raft_consensus.cc:482] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:03:56.468219 16716 raft_consensus.cc:3032] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Stepping down as leader of term 4
I20240501 08:03:56.468614 16792 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:56.470733 16789 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 5.
I20240501 08:03:56.472456 16791 raft_consensus.cc:1261] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:56.468678 16716 raft_consensus.cc:729] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:56.473821 16716 consensus_queue.cc:260] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:56.474208 16829 leader_election.cc:302] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:56.475081 16716 raft_consensus.cc:3037] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Advancing to term 5
I20240501 08:03:56.475255 17072 raft_consensus.cc:2781] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 5 FOLLOWER]: Leader election won for term 5
I20240501 08:03:56.475817 17072 raft_consensus.cc:686] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 5 LEADER]: Becoming Leader. State: Replica: 125c40043a7044939023b53e34e3a7e7, State: Running, Role: LEADER
I20240501 08:03:56.476640 17072 consensus_queue.cc:237] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } }
I20240501 08:03:56.483309 16716 raft_consensus.cc:2454] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 125c40043a7044939023b53e34e3a7e7 in term 5.
I20240501 08:03:56.468600 16790 raft_consensus.cc:504] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:56.484589 17072 consensus_queue.cc:1035] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20240501 08:03:56.491325 16790 leader_election.cc:288] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 4 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:56.487524 16792 raft_consensus.cc:504] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:56.491930 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:56.492645 16864 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:56.494216 16716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96373142392848c5b375f1f89cf2ec86" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:56.494706 16716 raft_consensus.cc:3032] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Stepping down as leader of term 3
I20240501 08:03:56.494964 16716 raft_consensus.cc:729] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:56.495576 16716 consensus_queue.cc:260] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:56.496500 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "125c40043a7044939023b53e34e3a7e7"
I20240501 08:03:56.496731 16716 raft_consensus.cc:3037] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Advancing to term 4
I20240501 08:03:56.497066 16865 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:56.498852 16715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321032c3a694400e820755e4cadd5815" candidate_uuid: "947186496a094f8885f3ad358cf8d03f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb35cf97ff15428fa584058ace6b6875"
I20240501 08:03:56.499418 16715 raft_consensus.cc:3032] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:03:56.499680 16715 raft_consensus.cc:729] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb35cf97ff15428fa584058ace6b6875, State: Running, Role: LEADER
I20240501 08:03:56.500397 16715 consensus_queue.cc:260] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:56.501695 16715 raft_consensus.cc:3037] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:03:56.503355 16716 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 4.
I20240501 08:03:56.504515 16755 leader_election.cc:302] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947186496a094f8885f3ad358cf8d03f, fb35cf97ff15428fa584058ace6b6875; no voters:
I20240501 08:03:56.505463 17031 raft_consensus.cc:2781] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Leader election won for term 4
I20240501 08:03:56.514670 16864 raft_consensus.cc:2454] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 4.
I20240501 08:03:56.516211 17031 raft_consensus.cc:686] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 4 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:56.517176 17031 consensus_queue.cc:237] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:56.517848 16614 catalog_manager.cc:5484] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 reported cstate change: term changed from 4 to 5, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 125c40043a7044939023b53e34e3a7e7 (127.15.168.131). New cstate: current_term: 5 leader_uuid: "125c40043a7044939023b53e34e3a7e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:56.513070 16865 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 3.
I20240501 08:03:56.522933 16757 leader_election.cc:302] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125c40043a7044939023b53e34e3a7e7, 947186496a094f8885f3ad358cf8d03f; no voters:
I20240501 08:03:56.523914 17080 raft_consensus.cc:2781] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:03:56.497088 16792 leader_election.cc:288] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [CANDIDATE]: Term 3 election: Requested vote from peers fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471), 125c40043a7044939023b53e34e3a7e7 (127.15.168.131:34079)
I20240501 08:03:56.526679 16715 raft_consensus.cc:2454] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947186496a094f8885f3ad358cf8d03f in term 3.
I20240501 08:03:56.530086 17080 raft_consensus.cc:686] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 3 LEADER]: Becoming Leader. State: Replica: 947186496a094f8885f3ad358cf8d03f, State: Running, Role: LEADER
I20240501 08:03:56.533545 17080 consensus_queue.cc:237] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } }
I20240501 08:03:56.552572 16612 catalog_manager.cc:5484] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 3 to 4, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 4 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:56.554124 16612 catalog_manager.cc:5484] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f reported cstate change: term changed from 2 to 3, leader changed from fb35cf97ff15428fa584058ace6b6875 (127.15.168.129) to 947186496a094f8885f3ad358cf8d03f (127.15.168.130). New cstate: current_term: 3 leader_uuid: "947186496a094f8885f3ad358cf8d03f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 } health_report { overall_health: UNKNOWN } } }
I20240501 08:03:56.779654 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:56.785743 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:56.786196 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
W20240501 08:03:56.881155 16738 debug-util.cc:398] Leaking SignalData structure 0x7b080004fae0 after lost signal to thread 16037
W20240501 08:03:56.881860 16738 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6f60 after lost signal to thread 16649
W20240501 08:03:56.882797 16738 debug-util.cc:398] Leaking SignalData structure 0x7b080010f900 after lost signal to thread 16741
W20240501 08:03:56.883656 16738 debug-util.cc:398] Leaking SignalData structure 0x7b0800121ee0 after lost signal to thread 16816
W20240501 08:03:56.884497 16738 debug-util.cc:398] Leaking SignalData structure 0x7b08001a5540 after lost signal to thread 16890
I20240501 08:03:56.929960 16865 raft_consensus.cc:1261] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:56.931771 17080 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:56.943977 16715 raft_consensus.cc:1261] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240501 08:03:56.945302 17080 consensus_queue.cc:1035] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:03:56.975131 16865 raft_consensus.cc:1261] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:56.976336 17080 consensus_queue.cc:1035] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "125c40043a7044939023b53e34e3a7e7" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:56.978982 16715 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 5 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240501 08:03:56.980288 17072 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:03:56.987690 16716 raft_consensus.cc:1261] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Refusing update from remote peer 947186496a094f8885f3ad358cf8d03f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240501 08:03:56.991961 17031 consensus_queue.cc:1035] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb35cf97ff15428fa584058ace6b6875" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:03:56.992686 16791 raft_consensus.cc:1261] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 5 FOLLOWER]: Refusing update from remote peer 125c40043a7044939023b53e34e3a7e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240501 08:03:56.994351 17062 consensus_queue.cc:1035] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947186496a094f8885f3ad358cf8d03f" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:03:57.786976 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:57.793457 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:57.793808 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:58.794658 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:58.799885 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:58.800228 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:03:59.800995 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:03:59.807893 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:03:59.808264 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:04:00.808966 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:04:00.814131 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:04:00.814568 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:04:01.815325 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:04:01.821278 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:04:01.821659 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:04:02.822429 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:04:02.827565 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:04:02.827951 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:04:03.828702 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:04:03.833804 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:04:03.834190 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:04:04.835193 16034 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240501 08:04:04.843649 16034 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240501 08:04:04.844033 16034 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240501 08:04:05.866883 16034 auto_leader_rebalancer-test.cc:376] The leader distribution is
I20240501 08:04:05.867436 16034 tablet_server.cc:178] TabletServer@127.15.168.129:0 shutting down...
W20240501 08:04:05.887356 16755 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20240501 08:04:05.891981 16755 consensus_peers.cc:485] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f -> Peer fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471): Couldn't send request to peer fb35cf97ff15428fa584058ace6b6875. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:05.896070 16034 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240501 08:04:05.896728 16034 tablet_replica.cc:330] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.897384 16034 raft_consensus.cc:2229] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:05.897995 16034 raft_consensus.cc:2258] T 44feef2d33b349bdb3cd2ae0389a5745 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.900265 16034 tablet_replica.cc:330] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.900779 16034 raft_consensus.cc:2229] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Raft consensus shutting down.
I20240501 08:04:05.901618 16034 raft_consensus.cc:2258] T 364c7a4942e24357bc5446f5d6ef7b01 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.903301 16034 tablet_replica.cc:330] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.903790 16034 raft_consensus.cc:2229] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:05.904165 16034 raft_consensus.cc:2258] T 9caf16c8ff3e4249b54bda04211f369d P fb35cf97ff15428fa584058ace6b6875 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.905738 16034 tablet_replica.cc:330] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.906180 16034 raft_consensus.cc:2229] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 5 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:05.906569 16034 raft_consensus.cc:2258] T fd7076d607314b098f3f1152781ae2f3 P fb35cf97ff15428fa584058ace6b6875 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.908144 16034 tablet_replica.cc:330] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.908599 16034 raft_consensus.cc:2229] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:05.908982 16034 raft_consensus.cc:2258] T 478b286082d34eecac6cc50fdcbe5e19 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.910537 16034 tablet_replica.cc:330] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.910959 16034 raft_consensus.cc:2229] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:05.911331 16034 raft_consensus.cc:2258] T 321032c3a694400e820755e4cadd5815 P fb35cf97ff15428fa584058ace6b6875 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.912748 16034 tablet_replica.cc:330] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.913182 16034 raft_consensus.cc:2229] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:05.913672 16034 raft_consensus.cc:2258] T 96373142392848c5b375f1f89cf2ec86 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.915167 16034 tablet_replica.cc:330] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.915602 16034 raft_consensus.cc:2229] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Raft consensus shutting down.
I20240501 08:04:05.916297 16034 raft_consensus.cc:2258] T d937a5aa00c74349a0953014727fdf2f P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.917833 16034 tablet_replica.cc:330] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.918274 16034 raft_consensus.cc:2229] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 2 LEADER]: Raft consensus shutting down.
I20240501 08:04:05.918984 16034 raft_consensus.cc:2258] T afd455adc27f480e97b44f042a946db3 P fb35cf97ff15428fa584058ace6b6875 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.920490 16034 tablet_replica.cc:330] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875: stopping tablet replica
I20240501 08:04:05.920876 16034 raft_consensus.cc:2229] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 4 LEADER]: Raft consensus shutting down.
I20240501 08:04:05.921666 16034 raft_consensus.cc:2258] T f67ff0cac9ea438196146004f9f9d740 P fb35cf97ff15428fa584058ace6b6875 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:05.971771 16034 tablet_server.cc:194] TabletServer@127.15.168.129:0 shutdown complete.
I20240501 08:04:05.989056 16034 tablet_server.cc:178] TabletServer@127.15.168.130:0 shutting down...
I20240501 08:04:06.012548 16034 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240501 08:04:06.013160 16034 tablet_replica.cc:330] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.013818 16034 raft_consensus.cc:2229] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.014262 16034 raft_consensus.cc:2258] T d937a5aa00c74349a0953014727fdf2f P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.016005 16034 tablet_replica.cc:330] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.016451 16034 raft_consensus.cc:2229] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.016855 16034 raft_consensus.cc:2258] T f67ff0cac9ea438196146004f9f9d740 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.018388 16034 tablet_replica.cc:330] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.018832 16034 raft_consensus.cc:2229] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.019208 16034 raft_consensus.cc:2258] T 364c7a4942e24357bc5446f5d6ef7b01 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.020726 16034 tablet_replica.cc:330] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.021163 16034 raft_consensus.cc:2229] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.021600 16034 raft_consensus.cc:2258] T 478b286082d34eecac6cc50fdcbe5e19 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.023108 16034 tablet_replica.cc:330] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.023548 16034 raft_consensus.cc:2229] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 4 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.024286 16034 raft_consensus.cc:2258] T 96373142392848c5b375f1f89cf2ec86 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.025772 16034 tablet_replica.cc:330] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.026243 16034 raft_consensus.cc:2229] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 3 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.026957 16034 raft_consensus.cc:2258] T 321032c3a694400e820755e4cadd5815 P 947186496a094f8885f3ad358cf8d03f [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.028412 16034 tablet_replica.cc:330] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.028833 16034 raft_consensus.cc:2229] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.029187 16034 raft_consensus.cc:2258] T 44feef2d33b349bdb3cd2ae0389a5745 P 947186496a094f8885f3ad358cf8d03f [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.030692 16034 tablet_replica.cc:330] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.031126 16034 raft_consensus.cc:2229] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 5 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.031539 16034 raft_consensus.cc:2258] T fd7076d607314b098f3f1152781ae2f3 P 947186496a094f8885f3ad358cf8d03f [term 5 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.032972 16034 tablet_replica.cc:330] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.033413 16034 raft_consensus.cc:2229] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 1 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.034075 16034 raft_consensus.cc:2258] T 9caf16c8ff3e4249b54bda04211f369d P 947186496a094f8885f3ad358cf8d03f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.035475 16034 tablet_replica.cc:330] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f: stopping tablet replica
I20240501 08:04:06.035876 16034 raft_consensus.cc:2229] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.036263 16034 raft_consensus.cc:2258] T afd455adc27f480e97b44f042a946db3 P 947186496a094f8885f3ad358cf8d03f [term 2 FOLLOWER]: Raft consensus is shut down!
W20240501 08:04:06.062496 16829 consensus_peers.cc:485] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 -> Peer fb35cf97ff15428fa584058ace6b6875 (127.15.168.129:42471): Couldn't send request to peer fb35cf97ff15428fa584058ace6b6875. Status: Network error: Client connection negotiation failed: client connection to 127.15.168.129:42471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:06.063782 16034 tablet_server.cc:194] TabletServer@127.15.168.130:0 shutdown complete.
I20240501 08:04:06.080360 16034 tablet_server.cc:178] TabletServer@127.15.168.131:0 shutting down...
I20240501 08:04:06.102208 16034 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240501 08:04:06.102807 16034 tablet_replica.cc:330] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.103389 16034 raft_consensus.cc:2229] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.103899 16034 raft_consensus.cc:2258] T 364c7a4942e24357bc5446f5d6ef7b01 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.105580 16034 tablet_replica.cc:330] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.106083 16034 raft_consensus.cc:2229] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.106463 16034 raft_consensus.cc:2258] T 9caf16c8ff3e4249b54bda04211f369d P 125c40043a7044939023b53e34e3a7e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.108033 16034 tablet_replica.cc:330] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.108489 16034 raft_consensus.cc:2229] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 5 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.109184 16034 raft_consensus.cc:2258] T fd7076d607314b098f3f1152781ae2f3 P 125c40043a7044939023b53e34e3a7e7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.110626 16034 tablet_replica.cc:330] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.111063 16034 raft_consensus.cc:2229] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 4 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.111742 16034 raft_consensus.cc:2258] T 44feef2d33b349bdb3cd2ae0389a5745 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.113221 16034 tablet_replica.cc:330] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.113693 16034 raft_consensus.cc:2229] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.114081 16034 raft_consensus.cc:2258] T d937a5aa00c74349a0953014727fdf2f P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.115553 16034 tablet_replica.cc:330] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.115983 16034 raft_consensus.cc:2229] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 2 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.116726 16034 raft_consensus.cc:2258] T 478b286082d34eecac6cc50fdcbe5e19 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.118335 16034 tablet_replica.cc:330] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.118777 16034 raft_consensus.cc:2229] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.119163 16034 raft_consensus.cc:2258] T 321032c3a694400e820755e4cadd5815 P 125c40043a7044939023b53e34e3a7e7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.120570 16034 tablet_replica.cc:330] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.120990 16034 raft_consensus.cc:2229] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.121446 16034 raft_consensus.cc:2258] T 96373142392848c5b375f1f89cf2ec86 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.122920 16034 tablet_replica.cc:330] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.123320 16034 raft_consensus.cc:2229] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.123689 16034 raft_consensus.cc:2258] T f67ff0cac9ea438196146004f9f9d740 P 125c40043a7044939023b53e34e3a7e7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.125099 16034 tablet_replica.cc:330] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7: stopping tablet replica
I20240501 08:04:06.125543 16034 raft_consensus.cc:2229] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:06.125927 16034 raft_consensus.cc:2258] T afd455adc27f480e97b44f042a946db3 P 125c40043a7044939023b53e34e3a7e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.151711 16034 tablet_server.cc:194] TabletServer@127.15.168.131:0 shutdown complete.
I20240501 08:04:06.168479 16034 master.cc:537] Master@127.15.168.190:40777 shutting down...
I20240501 08:04:06.187649 16034 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 1 LEADER]: Raft consensus shutting down.
I20240501 08:04:06.188180 16034 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:06.188607 16034 tablet_replica.cc:330] T 00000000000000000000000000000000 P e0b8e2c6c3a04323b71acd16e17c744c: stopping tablet replica
I20240501 08:04:06.224611 16034 master.cc:552] Master@127.15.168.190:40777 shutdown complete.
125c40043a7044939023b53e34e3a7e7 1
947186496a094f8885f3ad358cf8d03f 4
fb35cf97ff15428fa584058ace6b6875 5
125c40043a7044939023b53e34e3a7e7 3
947186496a094f8885f3ad358cf8d03f 4
fb35cf97ff15428fa584058ace6b6875 3
125c40043a7044939023b53e34e3a7e7 1
947186496a094f8885f3ad358cf8d03f 1
fb35cf97ff15428fa584058ace6b6875 8
125c40043a7044939023b53e34e3a7e7 3
947186496a094f8885f3ad358cf8d03f 3
fb35cf97ff15428fa584058ace6b6875 4
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28647 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20240501 08:04:06.263350 16034 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.168.190:41731
I20240501 08:04:06.264346 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:04:06.270100 17110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:06.270650 17111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:06.271469 17113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:06.272202 16034 server_base.cc:1034] running on GCE node
I20240501 08:04:06.273054 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:04:06.273227 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:04:06.273386 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550646273373 us; error 0 us; skew 500 ppm
I20240501 08:04:06.273816 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:04:06.276031 16034 webserver.cc:460] Webserver started at http://127.15.168.190:39537/ using document root <none> and password file <none>
I20240501 08:04:06.276443 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:04:06.276592 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:04:06.276808 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:04:06.277901 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/master-0-root/instance:
uuid: "b6991371d712420b8d49b3f6af6b123e"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.281922 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240501 08:04:06.284804 17118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.285482 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 08:04:06.285728 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/master-0-root
uuid: "b6991371d712420b8d49b3f6af6b123e"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.285981 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:04:06.301854 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:04:06.302903 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:04:06.336829 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.190:41731
I20240501 08:04:06.336915 17169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.190:41731 every 8 connection(s)
I20240501 08:04:06.340502 17170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.349594 17170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e: Bootstrap starting.
I20240501 08:04:06.353564 17170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:06.357308 17170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e: No bootstrap required, opened a new log
I20240501 08:04:06.359143 17170 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6991371d712420b8d49b3f6af6b123e" member_type: VOTER }
I20240501 08:04:06.359599 17170 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:06.359803 17170 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6991371d712420b8d49b3f6af6b123e, State: Initialized, Role: FOLLOWER
I20240501 08:04:06.360352 17170 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6991371d712420b8d49b3f6af6b123e" member_type: VOTER }
I20240501 08:04:06.360795 17170 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:04:06.361011 17170 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:04:06.361271 17170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:06.364853 17170 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6991371d712420b8d49b3f6af6b123e" member_type: VOTER }
I20240501 08:04:06.365330 17170 leader_election.cc:302] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6991371d712420b8d49b3f6af6b123e; no voters:
I20240501 08:04:06.366315 17170 leader_election.cc:288] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:04:06.366650 17173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:06.367782 17173 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [term 1 LEADER]: Becoming Leader. State: Replica: b6991371d712420b8d49b3f6af6b123e, State: Running, Role: LEADER
I20240501 08:04:06.368413 17173 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6991371d712420b8d49b3f6af6b123e" member_type: VOTER }
I20240501 08:04:06.368956 17170 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [sys.catalog]: configured and running, proceeding with master startup.
I20240501 08:04:06.372489 17174 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6991371d712420b8d49b3f6af6b123e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6991371d712420b8d49b3f6af6b123e" member_type: VOTER } }
I20240501 08:04:06.372676 17175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6991371d712420b8d49b3f6af6b123e. Latest consensus state: current_term: 1 leader_uuid: "b6991371d712420b8d49b3f6af6b123e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6991371d712420b8d49b3f6af6b123e" member_type: VOTER } }
I20240501 08:04:06.373131 17174 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [sys.catalog]: This master's current role is: LEADER
I20240501 08:04:06.373390 17175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e [sys.catalog]: This master's current role is: LEADER
I20240501 08:04:06.376046 17179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 08:04:06.382948 17179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 08:04:06.385495 16034 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240501 08:04:06.391453 17179 catalog_manager.cc:1347] Generated new cluster ID: b14348c076fd48cf9b725669690f7b29
I20240501 08:04:06.391713 17179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 08:04:06.403347 17179 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 08:04:06.404479 17179 catalog_manager.cc:1504] Loading token signing keys...
I20240501 08:04:06.421767 17179 catalog_manager.cc:5857] T 00000000000000000000000000000000 P b6991371d712420b8d49b3f6af6b123e: Generated new TSK 0
I20240501 08:04:06.422319 17179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 08:04:06.451391 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:04:06.458482 17191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:06.458516 17192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:06.459363 17194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:06.459955 16034 server_base.cc:1034] running on GCE node
I20240501 08:04:06.460767 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:04:06.460942 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:04:06.461084 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550646461068 us; error 0 us; skew 500 ppm
I20240501 08:04:06.461621 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:04:06.463733 16034 webserver.cc:460] Webserver started at http://127.15.168.129:42189/ using document root <none> and password file <none>
I20240501 08:04:06.464185 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:04:06.464363 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:04:06.464592 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:04:06.465646 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root/instance:
uuid: "fa9d3bc74a68481e9de647223cb0110e"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.469730 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240501 08:04:06.472754 17199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.473426 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 08:04:06.473670 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root
uuid: "fa9d3bc74a68481e9de647223cb0110e"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.473925 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:04:06.493355 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:04:06.494464 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:04:06.495819 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:04:06.497927 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:04:06.498112 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.498333 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:04:06.498492 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.534483 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.129:44541
I20240501 08:04:06.534585 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.129:44541 every 8 connection(s)
I20240501 08:04:06.538652 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:04:06.546496 17266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:06.547463 17267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:06.551349 16034 server_base.cc:1034] running on GCE node
I20240501 08:04:06.551528 17262 heartbeater.cc:347] Connected to a master server at 127.15.168.190:41731
I20240501 08:04:06.551898 17262 heartbeater.cc:464] Registering TS with master...
W20240501 08:04:06.552160 17269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:06.552675 17262 heartbeater.cc:511] Master 127.15.168.190:41731 requested a full tablet report, sending...
I20240501 08:04:06.552986 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:04:06.553181 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:04:06.553361 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550646553345 us; error 0 us; skew 500 ppm
I20240501 08:04:06.554086 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:04:06.554527 17135 ts_manager.cc:194] Registered new tserver with Master: fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:06.556726 17135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53870
I20240501 08:04:06.557924 16034 webserver.cc:460] Webserver started at http://127.15.168.130:39349/ using document root <none> and password file <none>
I20240501 08:04:06.558559 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:04:06.558794 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:04:06.559108 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:04:06.560178 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root/instance:
uuid: "cdccadabd9154ea69e15405d75301fc1"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.564196 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20240501 08:04:06.567166 17274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.567811 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:04:06.568068 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root
uuid: "cdccadabd9154ea69e15405d75301fc1"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.568328 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:04:06.578372 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:04:06.579416 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:04:06.580672 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:04:06.582718 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:04:06.582901 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.583117 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:04:06.583261 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.618837 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.130:33757
I20240501 08:04:06.618932 17336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.130:33757 every 8 connection(s)
I20240501 08:04:06.623169 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:04:06.630630 17340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:06.631839 17341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:06.633096 16034 server_base.cc:1034] running on GCE node
W20240501 08:04:06.633864 17343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:06.634763 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:04:06.635025 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:04:06.635200 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550646635185 us; error 0 us; skew 500 ppm
I20240501 08:04:06.635480 17337 heartbeater.cc:347] Connected to a master server at 127.15.168.190:41731
I20240501 08:04:06.635788 17337 heartbeater.cc:464] Registering TS with master...
I20240501 08:04:06.635824 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:04:06.636493 17337 heartbeater.cc:511] Master 127.15.168.190:41731 requested a full tablet report, sending...
I20240501 08:04:06.638414 16034 webserver.cc:460] Webserver started at http://127.15.168.131:33411/ using document root <none> and password file <none>
I20240501 08:04:06.638410 17135 ts_manager.cc:194] Registered new tserver with Master: cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:06.639014 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:04:06.639230 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:04:06.639458 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:04:06.639871 17135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53872
I20240501 08:04:06.640612 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root/instance:
uuid: "5655c2e9168d4eeaa212db8874c2efa5"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.644668 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20240501 08:04:06.647604 17348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.648244 16034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:04:06.648495 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root
uuid: "5655c2e9168d4eeaa212db8874c2efa5"
format_stamp: "Formatted at 2024-05-01 08:04:06 on dist-test-slave-g8jl"
I20240501 08:04:06.648766 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:04:06.666306 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:04:06.667351 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:04:06.668656 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:04:06.670879 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:04:06.671061 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.671273 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:04:06.671422 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:06.707389 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.131:43915
I20240501 08:04:06.707489 17410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.131:43915 every 8 connection(s)
I20240501 08:04:06.719302 17411 heartbeater.cc:347] Connected to a master server at 127.15.168.190:41731
I20240501 08:04:06.719648 17411 heartbeater.cc:464] Registering TS with master...
I20240501 08:04:06.720327 17411 heartbeater.cc:511] Master 127.15.168.190:41731 requested a full tablet report, sending...
I20240501 08:04:06.721980 17135 ts_manager.cc:194] Registered new tserver with Master: 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:06.722069 16034 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.011914825s
I20240501 08:04:06.723227 17135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53884
I20240501 08:04:06.724293 16034 test_util.cc:268] Using random seed: -297838928
I20240501 08:04:06.745121 17135 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53888:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240501 08:04:06.751145 17135 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240501 08:04:06.943887 17376 tablet_service.cc:1467] Processing CreateTablet for tablet 8510a331dab34f468a303fa5b0054421 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240501 08:04:06.945431 17376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8510a331dab34f468a303fa5b0054421. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.948671 17302 tablet_service.cc:1467] Processing CreateTablet for tablet 8510a331dab34f468a303fa5b0054421 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240501 08:04:06.950007 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8510a331dab34f468a303fa5b0054421. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.953431 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 68f430c048c149c889744933e9c6a2ac (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240501 08:04:06.954669 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f430c048c149c889744933e9c6a2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.957594 17227 tablet_service.cc:1467] Processing CreateTablet for tablet 8510a331dab34f468a303fa5b0054421 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240501 08:04:06.958317 17300 tablet_service.cc:1467] Processing CreateTablet for tablet e5e141ff8f064eb480d6e2174c541e42 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240501 08:04:06.958813 17227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8510a331dab34f468a303fa5b0054421. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.959316 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 3e00624b7db1409890298f5a6d88a575 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240501 08:04:06.959961 17299 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8c13df28b14b428733c4c25c234a6b (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240501 08:04:06.960517 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e00624b7db1409890298f5a6d88a575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.962323 17226 tablet_service.cc:1467] Processing CreateTablet for tablet 68f430c048c149c889744933e9c6a2ac (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240501 08:04:06.963577 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f430c048c149c889744933e9c6a2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.963368 17297 tablet_service.cc:1467] Processing CreateTablet for tablet 068fca00dea44c6fbac5bc5365a66a4b (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240501 08:04:06.964460 17220 tablet_service.cc:1467] Processing CreateTablet for tablet 2250a60c258544668fa164fcf11634e4 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240501 08:04:06.965682 17220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2250a60c258544668fa164fcf11634e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.966310 17225 tablet_service.cc:1467] Processing CreateTablet for tablet e5e141ff8f064eb480d6e2174c541e42 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240501 08:04:06.967571 17296 tablet_service.cc:1467] Processing CreateTablet for tablet b41080472c7646c68dd171e816059375 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240501 08:04:06.968492 17301 tablet_service.cc:1467] Processing CreateTablet for tablet 68f430c048c149c889744933e9c6a2ac (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240501 08:04:06.961172 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8c13df28b14b428733c4c25c234a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.962328 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 068fca00dea44c6fbac5bc5365a66a4b (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240501 08:04:06.966325 17295 tablet_service.cc:1467] Processing CreateTablet for tablet 2250a60c258544668fa164fcf11634e4 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240501 08:04:06.967608 17294 tablet_service.cc:1467] Processing CreateTablet for tablet f3fd46911e2c4ac8a341f1d910bc978e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240501 08:04:06.962320 17298 tablet_service.cc:1467] Processing CreateTablet for tablet 3e00624b7db1409890298f5a6d88a575 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240501 08:04:06.970983 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 19b58febdb8147b699a76df8ce1cb923 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240501 08:04:06.967494 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e141ff8f064eb480d6e2174c541e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.963318 17221 tablet_service.cc:1467] Processing CreateTablet for tablet b41080472c7646c68dd171e816059375 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240501 08:04:06.971096 17293 tablet_service.cc:1467] Processing CreateTablet for tablet 19b58febdb8147b699a76df8ce1cb923 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240501 08:04:06.972525 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2250a60c258544668fa164fcf11634e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.969676 17219 tablet_service.cc:1467] Processing CreateTablet for tablet f3fd46911e2c4ac8a341f1d910bc978e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240501 08:04:06.973435 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19b58febdb8147b699a76df8ce1cb923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.974236 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3fd46911e2c4ac8a341f1d910bc978e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.975155 17301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f430c048c149c889744933e9c6a2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.976262 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b41080472c7646c68dd171e816059375. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.977031 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e141ff8f064eb480d6e2174c541e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.968736 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8c13df28b14b428733c4c25c234a6b (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240501 08:04:06.978442 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e00624b7db1409890298f5a6d88a575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.980129 17297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 068fca00dea44c6fbac5bc5365a66a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.972069 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 068fca00dea44c6fbac5bc5365a66a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.982800 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b41080472c7646c68dd171e816059375. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.983675 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8c13df28b14b428733c4c25c234a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.984302 17219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3fd46911e2c4ac8a341f1d910bc978e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:06.985617 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19b58febdb8147b699a76df8ce1cb923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.002631 17299 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4d63ea4cc54bb1962a1fe4c656ce95 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240501 08:04:07.003908 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f4d63ea4cc54bb1962a1fe4c656ce95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.021414 17374 tablet_service.cc:1467] Processing CreateTablet for tablet e5e141ff8f064eb480d6e2174c541e42 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240501 08:04:07.022737 17374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e141ff8f064eb480d6e2174c541e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.025063 17299 tablet_service.cc:1467] Processing CreateTablet for tablet 384fc32cbcd0455b9cce4f5e71f9a626 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240501 08:04:07.032112 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384fc32cbcd0455b9cce4f5e71f9a626. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.038955 17373 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8c13df28b14b428733c4c25c234a6b (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240501 08:04:07.040223 17373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8c13df28b14b428733c4c25c234a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.053309 17372 tablet_service.cc:1467] Processing CreateTablet for tablet 3e00624b7db1409890298f5a6d88a575 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240501 08:04:07.055297 17371 tablet_service.cc:1467] Processing CreateTablet for tablet 068fca00dea44c6fbac5bc5365a66a4b (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240501 08:04:07.056427 17299 tablet_service.cc:1467] Processing CreateTablet for tablet d636a072366c4776aecc4b164749d410 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240501 08:04:07.058123 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d636a072366c4776aecc4b164749d410. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.059624 17295 tablet_service.cc:1467] Processing CreateTablet for tablet 1189be88e7b341918c1fa148b2d77030 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240501 08:04:07.060539 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e00624b7db1409890298f5a6d88a575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.060822 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1189be88e7b341918c1fa148b2d77030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.061960 17371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 068fca00dea44c6fbac5bc5365a66a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.065433 17370 tablet_service.cc:1467] Processing CreateTablet for tablet b41080472c7646c68dd171e816059375 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240501 08:04:07.068780 17369 tablet_service.cc:1467] Processing CreateTablet for tablet 2250a60c258544668fa164fcf11634e4 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240501 08:04:07.070791 17367 tablet_service.cc:1467] Processing CreateTablet for tablet 19b58febdb8147b699a76df8ce1cb923 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240501 08:04:07.072744 17368 tablet_service.cc:1467] Processing CreateTablet for tablet f3fd46911e2c4ac8a341f1d910bc978e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240501 08:04:07.075057 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b41080472c7646c68dd171e816059375. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.076576 17302 tablet_service.cc:1467] Processing CreateTablet for tablet 298e75ee3b0345ea8766285d1ef4e09e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240501 08:04:07.077723 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298e75ee3b0345ea8766285d1ef4e09e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.087777 17367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19b58febdb8147b699a76df8ce1cb923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.093537 17369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2250a60c258544668fa164fcf11634e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.097533 17368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3fd46911e2c4ac8a341f1d910bc978e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.114280 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 384fc32cbcd0455b9cce4f5e71f9a626 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240501 08:04:07.115082 17226 tablet_service.cc:1467] Processing CreateTablet for tablet d636a072366c4776aecc4b164749d410 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240501 08:04:07.124794 17225 tablet_service.cc:1467] Processing CreateTablet for tablet 298e75ee3b0345ea8766285d1ef4e09e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240501 08:04:07.115909 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 1189be88e7b341918c1fa148b2d77030 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240501 08:04:07.120501 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4d63ea4cc54bb1962a1fe4c656ce95 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240501 08:04:07.127926 17294 tablet_service.cc:1467] Processing CreateTablet for tablet f5cd3725c0f14e00a00542d3285b2d9d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240501 08:04:07.121897 17426 tablet_bootstrap.cc:492] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:07.128419 17221 tablet_service.cc:1467] Processing CreateTablet for tablet fe77234745b44d4983ec4508289a3a25 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240501 08:04:07.129137 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5cd3725c0f14e00a00542d3285b2d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.131664 17295 tablet_service.cc:1467] Processing CreateTablet for tablet 81960706d26c481ba0b5441075d45c30 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240501 08:04:07.132781 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f4d63ea4cc54bb1962a1fe4c656ce95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.123718 17372 tablet_service.cc:1467] Processing CreateTablet for tablet 384fc32cbcd0455b9cce4f5e71f9a626 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240501 08:04:07.133978 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384fc32cbcd0455b9cce4f5e71f9a626. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.131352 17298 tablet_service.cc:1467] Processing CreateTablet for tablet 23486d7c47664dc89019c4d92d25e82d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240501 08:04:07.115621 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384fc32cbcd0455b9cce4f5e71f9a626. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.125572 17299 tablet_service.cc:1467] Processing CreateTablet for tablet fe77234745b44d4983ec4508289a3a25 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240501 08:04:07.132805 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81960706d26c481ba0b5441075d45c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.138620 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe77234745b44d4983ec4508289a3a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.138954 17427 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:07.135587 17220 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4d63ea4cc54bb1962a1fe4c656ce95 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240501 08:04:07.131893 17293 tablet_service.cc:1467] Processing CreateTablet for tablet 37a8e259653a4dc28c1c6875ec8b520a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240501 08:04:07.142494 17428 tablet_bootstrap.cc:492] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:07.147815 17426 tablet_bootstrap.cc:654] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.148038 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1189be88e7b341918c1fa148b2d77030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.150185 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298e75ee3b0345ea8766285d1ef4e09e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.152088 17302 tablet_service.cc:1467] Processing CreateTablet for tablet 57ca37e5ca8844c68b9522e5d05bfc3e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240501 08:04:07.154069 17296 tablet_service.cc:1467] Processing CreateTablet for tablet 4eb40f0013fc419fb843b002645ed74d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240501 08:04:07.159744 17427 tablet_bootstrap.cc:654] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.134980 17301 tablet_service.cc:1467] Processing CreateTablet for tablet 5646a19da7ae4c4eafcc2532ee9326c3 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240501 08:04:07.166260 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23486d7c47664dc89019c4d92d25e82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.166612 17297 tablet_service.cc:1467] Processing CreateTablet for tablet a732d904a75041c29d28ec6510e02a32 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240501 08:04:07.167440 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d636a072366c4776aecc4b164749d410. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.169296 17373 tablet_service.cc:1467] Processing CreateTablet for tablet 1189be88e7b341918c1fa148b2d77030 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240501 08:04:07.170583 17373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1189be88e7b341918c1fa148b2d77030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.171947 17369 tablet_service.cc:1467] Processing CreateTablet for tablet 298e75ee3b0345ea8766285d1ef4e09e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240501 08:04:07.184902 17369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298e75ee3b0345ea8766285d1ef4e09e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.149183 17370 tablet_service.cc:1467] Processing CreateTablet for tablet d636a072366c4776aecc4b164749d410 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240501 08:04:07.172108 17374 tablet_service.cc:1467] Processing CreateTablet for tablet fe77234745b44d4983ec4508289a3a25 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240501 08:04:07.173102 17220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f4d63ea4cc54bb1962a1fe4c656ce95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.174650 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 37a8e259653a4dc28c1c6875ec8b520a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240501 08:04:07.188721 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a8e259653a4dc28c1c6875ec8b520a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.195003 17376 tablet_service.cc:1467] Processing CreateTablet for tablet f5cd3725c0f14e00a00542d3285b2d9d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240501 08:04:07.186425 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d636a072366c4776aecc4b164749d410. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.177441 17300 tablet_service.cc:1467] Processing CreateTablet for tablet c003e520523e4df5866a64c4612d1970 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240501 08:04:07.179138 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 5646a19da7ae4c4eafcc2532ee9326c3 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240501 08:04:07.202637 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5646a19da7ae4c4eafcc2532ee9326c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.181900 17371 tablet_service.cc:1467] Processing CreateTablet for tablet 37a8e259653a4dc28c1c6875ec8b520a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240501 08:04:07.203538 17371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a8e259653a4dc28c1c6875ec8b520a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.172993 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a8e259653a4dc28c1c6875ec8b520a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.204294 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c003e520523e4df5866a64c4612d1970. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.183558 17227 tablet_service.cc:1467] Processing CreateTablet for tablet 23486d7c47664dc89019c4d92d25e82d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240501 08:04:07.206861 17227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23486d7c47664dc89019c4d92d25e82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.210261 17368 tablet_service.cc:1467] Processing CreateTablet for tablet 5646a19da7ae4c4eafcc2532ee9326c3 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240501 08:04:07.175662 17428 tablet_bootstrap.cc:654] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.181321 17219 tablet_service.cc:1467] Processing CreateTablet for tablet f5cd3725c0f14e00a00542d3285b2d9d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240501 08:04:07.212369 17219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5cd3725c0f14e00a00542d3285b2d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.205503 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eb40f0013fc419fb843b002645ed74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.217715 17301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5646a19da7ae4c4eafcc2532ee9326c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.211467 17368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5646a19da7ae4c4eafcc2532ee9326c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.221329 17426 tablet_bootstrap.cc:492] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:07.221798 17426 ts_tablet_manager.cc:1367] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.100s user 0.013s sys 0.000s
I20240501 08:04:07.224382 17426 raft_consensus.cc:348] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:07.230389 17374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe77234745b44d4983ec4508289a3a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.231457 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ca37e5ca8844c68b9522e5d05bfc3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.232125 17297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a732d904a75041c29d28ec6510e02a32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.225108 17426 raft_consensus.cc:374] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.232605 17426 raft_consensus.cc:729] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.233518 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe77234745b44d4983ec4508289a3a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.233331 17426 consensus_queue.cc:260] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:07.246017 17375 tablet_service.cc:1467] Processing CreateTablet for tablet a732d904a75041c29d28ec6510e02a32 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240501 08:04:07.247401 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 81960706d26c481ba0b5441075d45c30 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240501 08:04:07.248570 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81960706d26c481ba0b5441075d45c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.249492 17376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5cd3725c0f14e00a00542d3285b2d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.251174 17225 tablet_service.cc:1467] Processing CreateTablet for tablet 4eb40f0013fc419fb843b002645ed74d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240501 08:04:07.252310 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eb40f0013fc419fb843b002645ed74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.256201 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 57ca37e5ca8844c68b9522e5d05bfc3e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240501 08:04:07.276878 17302 tablet_service.cc:1467] Processing CreateTablet for tablet 0d77fc1fa8a74485a5698b48b0fa2472 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240501 08:04:07.278264 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d77fc1fa8a74485a5698b48b0fa2472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.279353 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ca37e5ca8844c68b9522e5d05bfc3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.296064 17219 tablet_service.cc:1467] Processing CreateTablet for tablet c003e520523e4df5866a64c4612d1970 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240501 08:04:07.298667 17226 tablet_service.cc:1467] Processing CreateTablet for tablet 0d77fc1fa8a74485a5698b48b0fa2472 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240501 08:04:07.301520 17372 tablet_service.cc:1467] Processing CreateTablet for tablet 81960706d26c481ba0b5441075d45c30 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240501 08:04:07.307453 17370 tablet_service.cc:1467] Processing CreateTablet for tablet 57ca37e5ca8844c68b9522e5d05bfc3e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240501 08:04:07.308946 17227 tablet_service.cc:1467] Processing CreateTablet for tablet a87f059d40b441d5ad84b2e410829e61 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240501 08:04:07.309741 17373 tablet_service.cc:1467] Processing CreateTablet for tablet 23486d7c47664dc89019c4d92d25e82d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240501 08:04:07.310374 17219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c003e520523e4df5866a64c4612d1970. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.315353 17298 tablet_service.cc:1467] Processing CreateTablet for tablet a87f059d40b441d5ad84b2e410829e61 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240501 08:04:07.328567 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87f059d40b441d5ad84b2e410829e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.329377 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a732d904a75041c29d28ec6510e02a32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.326838 17337 heartbeater.cc:503] Master 127.15.168.190:41731 was elected leader, sending a full tablet report...
I20240501 08:04:07.331454 17367 tablet_service.cc:1467] Processing CreateTablet for tablet 4eb40f0013fc419fb843b002645ed74d (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240501 08:04:07.334195 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d77fc1fa8a74485a5698b48b0fa2472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.335068 17227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87f059d40b441d5ad84b2e410829e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.338428 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 38871d6cbe1846c89b09302d47294f87 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240501 08:04:07.347994 17428 tablet_bootstrap.cc:492] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:07.348160 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38871d6cbe1846c89b09302d47294f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.348466 17428 ts_tablet_manager.cc:1367] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.206s user 0.017s sys 0.015s
I20240501 08:04:07.349299 17218 tablet_service.cc:1467] Processing CreateTablet for tablet a732d904a75041c29d28ec6510e02a32 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240501 08:04:07.350458 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a732d904a75041c29d28ec6510e02a32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.346910 17224 tablet_service.cc:1467] Processing CreateTablet for tablet c7567a8d257c4c908317642193d10087 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240501 08:04:07.353979 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7567a8d257c4c908317642193d10087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.355712 17294 tablet_service.cc:1467] Processing CreateTablet for tablet 38871d6cbe1846c89b09302d47294f87 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240501 08:04:07.356863 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38871d6cbe1846c89b09302d47294f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.358853 17301 tablet_service.cc:1467] Processing CreateTablet for tablet c7567a8d257c4c908317642193d10087 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240501 08:04:07.359979 17301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7567a8d257c4c908317642193d10087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.365954 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81960706d26c481ba0b5441075d45c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.366205 17295 tablet_service.cc:1467] Processing CreateTablet for tablet e4b47c5f05da46bc9d9b5b4b202c1210 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240501 08:04:07.367345 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b47c5f05da46bc9d9b5b4b202c1210. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.371362 17427 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:07.371796 17428 raft_consensus.cc:348] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.372566 17428 raft_consensus.cc:374] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.372864 17428 raft_consensus.cc:729] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.373791 17373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23486d7c47664dc89019c4d92d25e82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.375512 17302 tablet_service.cc:1467] Processing CreateTablet for tablet b993df33624a4500a73b663ca80d75f7 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240501 08:04:07.375895 17371 tablet_service.cc:1467] Processing CreateTablet for tablet c003e520523e4df5866a64c4612d1970 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240501 08:04:07.376660 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b993df33624a4500a73b663ca80d75f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.377084 17371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c003e520523e4df5866a64c4612d1970. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.378230 17426 ts_tablet_manager.cc:1398] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.156s user 0.020s sys 0.011s
I20240501 08:04:07.379115 17426 tablet_bootstrap.cc:492] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
W20240501 08:04:07.381683 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:07.382102 17220 tablet_service.cc:1467] Processing CreateTablet for tablet e4b47c5f05da46bc9d9b5b4b202c1210 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240501 08:04:07.383236 17220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b47c5f05da46bc9d9b5b4b202c1210. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.371829 17427 ts_tablet_manager.cc:1367] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.233s user 0.035s sys 0.056s
I20240501 08:04:07.387151 17221 tablet_service.cc:1467] Processing CreateTablet for tablet b993df33624a4500a73b663ca80d75f7 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240501 08:04:07.388437 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b993df33624a4500a73b663ca80d75f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.393093 17300 tablet_service.cc:1467] Processing CreateTablet for tablet 44468f0ec3d94b97b8ea60830d3b7393 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240501 08:04:07.394248 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44468f0ec3d94b97b8ea60830d3b7393. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.394727 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 44468f0ec3d94b97b8ea60830d3b7393 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240501 08:04:07.394876 17427 raft_consensus.cc:348] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.395619 17427 raft_consensus.cc:374] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.395874 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44468f0ec3d94b97b8ea60830d3b7393. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.396355 17428 consensus_queue.cc:260] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.395803 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 0d9db161d31444f9a07bf586dff9a45a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240501 08:04:07.396900 17373 tablet_service.cc:1467] Processing CreateTablet for tablet 0d77fc1fa8a74485a5698b48b0fa2472 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240501 08:04:07.399161 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 726057013c1e48f985e1fd55e502c0e0 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240501 08:04:07.399281 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d9db161d31444f9a07bf586dff9a45a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.400292 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726057013c1e48f985e1fd55e502c0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.401001 17373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d77fc1fa8a74485a5698b48b0fa2472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.405784 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57ca37e5ca8844c68b9522e5d05bfc3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.402249 17225 tablet_service.cc:1467] Processing CreateTablet for tablet 85c1e10c92f744c3a22dec7eecae1040 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240501 08:04:07.411437 17367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eb40f0013fc419fb843b002645ed74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.411826 17374 tablet_service.cc:1467] Processing CreateTablet for tablet 38871d6cbe1846c89b09302d47294f87 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240501 08:04:07.412917 17374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38871d6cbe1846c89b09302d47294f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.406997 17427 raft_consensus.cc:729] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.420280 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c1e10c92f744c3a22dec7eecae1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.427682 17375 tablet_service.cc:1467] Processing CreateTablet for tablet c7567a8d257c4c908317642193d10087 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240501 08:04:07.429085 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7567a8d257c4c908317642193d10087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.416298 17369 tablet_service.cc:1467] Processing CreateTablet for tablet a87f059d40b441d5ad84b2e410829e61 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240501 08:04:07.488165 17369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87f059d40b441d5ad84b2e410829e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.434748 17426 tablet_bootstrap.cc:654] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.446642 17372 tablet_service.cc:1467] Processing CreateTablet for tablet e4b47c5f05da46bc9d9b5b4b202c1210 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240501 08:04:07.458809 17371 tablet_service.cc:1467] Processing CreateTablet for tablet b993df33624a4500a73b663ca80d75f7 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240501 08:04:07.460322 17373 tablet_service.cc:1467] Processing CreateTablet for tablet 44468f0ec3d94b97b8ea60830d3b7393 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240501 08:04:07.470738 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 990374acc0c34c788d07bca6603f2e1e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240501 08:04:07.474675 17370 tablet_service.cc:1467] Processing CreateTablet for tablet 85c1e10c92f744c3a22dec7eecae1040 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240501 08:04:07.468619 17368 tablet_service.cc:1467] Processing CreateTablet for tablet 0d9db161d31444f9a07bf586dff9a45a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240501 08:04:07.486673 17293 tablet_service.cc:1467] Processing CreateTablet for tablet 0d9db161d31444f9a07bf586dff9a45a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240501 08:04:07.486917 17299 tablet_service.cc:1467] Processing CreateTablet for tablet 726057013c1e48f985e1fd55e502c0e0 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240501 08:04:07.506119 17428 ts_tablet_manager.cc:1398] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.137s user 0.032s sys 0.049s
I20240501 08:04:07.507895 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c1e10c92f744c3a22dec7eecae1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.508527 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b47c5f05da46bc9d9b5b4b202c1210. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.509568 17411 heartbeater.cc:503] Master 127.15.168.190:41731 was elected leader, sending a full tablet report...
I20240501 08:04:07.509727 17376 tablet_service.cc:1467] Processing CreateTablet for tablet 726057013c1e48f985e1fd55e502c0e0 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240501 08:04:07.510465 17428 tablet_bootstrap.cc:492] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:07.510901 17368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d9db161d31444f9a07bf586dff9a45a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.513456 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726057013c1e48f985e1fd55e502c0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.517200 17428 tablet_bootstrap.cc:654] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.517457 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990374acc0c34c788d07bca6603f2e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.519771 17374 tablet_service.cc:1467] Processing CreateTablet for tablet 990374acc0c34c788d07bca6603f2e1e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240501 08:04:07.533077 17219 tablet_service.cc:1467] Processing CreateTablet for tablet 64a7c27c8f0b41dca4f760d78b7d94fc (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240501 08:04:07.541829 17226 tablet_service.cc:1467] Processing CreateTablet for tablet e5f504d6ad6b4b77927fb216e12347e8 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240501 08:04:07.542270 17225 tablet_service.cc:1467] Processing CreateTablet for tablet c3708b02025940e589e0e97e994b0c70 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240501 08:04:07.527181 17376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726057013c1e48f985e1fd55e502c0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.545167 17221 tablet_service.cc:1467] Processing CreateTablet for tablet 0274d08768ff4a1892e81b59d1677f2a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240501 08:04:07.547492 17369 tablet_service.cc:1467] Processing CreateTablet for tablet 64a7c27c8f0b41dca4f760d78b7d94fc (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240501 08:04:07.564882 17374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990374acc0c34c788d07bca6603f2e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.564487 17262 heartbeater.cc:503] Master 127.15.168.190:41731 was elected leader, sending a full tablet report...
I20240501 08:04:07.568113 17369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64a7c27c8f0b41dca4f760d78b7d94fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.564297 17301 tablet_service.cc:1467] Processing CreateTablet for tablet 64a7c27c8f0b41dca4f760d78b7d94fc (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240501 08:04:07.569566 17301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64a7c27c8f0b41dca4f760d78b7d94fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.562412 17298 tablet_service.cc:1467] Processing CreateTablet for tablet 85c1e10c92f744c3a22dec7eecae1040 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240501 08:04:07.571214 17367 tablet_service.cc:1467] Processing CreateTablet for tablet e5f504d6ad6b4b77927fb216e12347e8 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240501 08:04:07.571621 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c1e10c92f744c3a22dec7eecae1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.571152 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3708b02025940e589e0e97e994b0c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.572389 17367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f504d6ad6b4b77927fb216e12347e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.573863 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0274d08768ff4a1892e81b59d1677f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.578272 17373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44468f0ec3d94b97b8ea60830d3b7393. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.580989 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 66624ff54d8844e3bf88af88be895fd3 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240501 08:04:07.582300 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66624ff54d8844e3bf88af88be895fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.583823 17219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64a7c27c8f0b41dca4f760d78b7d94fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.495982 17427 consensus_queue.cc:260] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.592370 17297 tablet_service.cc:1467] Processing CreateTablet for tablet 990374acc0c34c788d07bca6603f2e1e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240501 08:04:07.594527 17302 tablet_service.cc:1467] Processing CreateTablet for tablet e5f504d6ad6b4b77927fb216e12347e8 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
W20240501 08:04:07.605398 17121 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.168.130:33757, user_credentials={real_user=slave}} blocked reactor thread for 59081.9us
I20240501 08:04:07.607626 17426 tablet_bootstrap.cc:492] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:07.612807 17432 raft_consensus.cc:482] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:07.618191 17432 raft_consensus.cc:504] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:07.619220 17372 tablet_service.cc:1467] Processing CreateTablet for tablet 66624ff54d8844e3bf88af88be895fd3 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240501 08:04:07.620535 17432 leader_election.cc:288] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915), fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:07.617466 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d9db161d31444f9a07bf586dff9a45a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.620448 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66624ff54d8844e3bf88af88be895fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.630424 17427 ts_tablet_manager.cc:1398] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.242s user 0.003s sys 0.005s
I20240501 08:04:07.630628 17426 ts_tablet_manager.cc:1367] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.251s user 0.017s sys 0.004s
I20240501 08:04:07.646857 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f504d6ad6b4b77927fb216e12347e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.650252 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f504d6ad6b4b77927fb216e12347e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.656400 17427 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:07.645071 17426 raft_consensus.cc:348] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.662118 17426 raft_consensus.cc:374] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.662384 17427 tablet_bootstrap.cc:654] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.662420 17426 raft_consensus.cc:729] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.669540 17426 consensus_queue.cc:260] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.671852 17426 ts_tablet_manager.cc:1398] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20240501 08:04:07.672698 17426 tablet_bootstrap.cc:492] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:07.676877 17296 tablet_service.cc:1467] Processing CreateTablet for tablet 66624ff54d8844e3bf88af88be895fd3 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240501 08:04:07.682003 17428 tablet_bootstrap.cc:492] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:07.682384 17433 raft_consensus.cc:482] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:07.682830 17433 raft_consensus.cc:504] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.685132 17433 leader_election.cc:288] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:07.682487 17428 ts_tablet_manager.cc:1367] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.172s user 0.011s sys 0.018s
I20240501 08:04:07.690790 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 4437b60cbdcd4b7d850f3534a8845e92 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240501 08:04:07.691774 17371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b993df33624a4500a73b663ca80d75f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.691963 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4437b60cbdcd4b7d850f3534a8845e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.695443 17428 raft_consensus.cc:348] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.696163 17428 raft_consensus.cc:374] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.696426 17428 raft_consensus.cc:729] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.697213 17294 tablet_service.cc:1467] Processing CreateTablet for tablet 0274d08768ff4a1892e81b59d1677f2a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240501 08:04:07.700388 17300 tablet_service.cc:1467] Processing CreateTablet for tablet 4437b60cbdcd4b7d850f3534a8845e92 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240501 08:04:07.701773 17297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990374acc0c34c788d07bca6603f2e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.710158 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0274d08768ff4a1892e81b59d1677f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.709991 17376 tablet_service.cc:1467] Processing CreateTablet for tablet c3708b02025940e589e0e97e994b0c70 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240501 08:04:07.710721 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4437b60cbdcd4b7d850f3534a8845e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.711148 17376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3708b02025940e589e0e97e994b0c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.713559 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66624ff54d8844e3bf88af88be895fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.716094 17426 tablet_bootstrap.cc:654] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.697077 17428 consensus_queue.cc:260] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.726613 17370 tablet_service.cc:1467] Processing CreateTablet for tablet 0274d08768ff4a1892e81b59d1677f2a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240501 08:04:07.727849 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0274d08768ff4a1892e81b59d1677f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.741171 17298 tablet_service.cc:1467] Processing CreateTablet for tablet c3708b02025940e589e0e97e994b0c70 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240501 08:04:07.742589 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3708b02025940e589e0e97e994b0c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.765154 17374 tablet_service.cc:1467] Processing CreateTablet for tablet 4437b60cbdcd4b7d850f3534a8845e92 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20240501 08:04:07.746197 17121 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.168.129:44541, user_credentials={real_user=slave}} blocked reactor thread for 48421.9us
I20240501 08:04:07.766371 17428 ts_tablet_manager.cc:1398] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.079s user 0.005s sys 0.000s
I20240501 08:04:07.766422 17374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4437b60cbdcd4b7d850f3534a8845e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.767746 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa513fcceeb4a15b0a193ddf00a6e62 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240501 08:04:07.769059 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa513fcceeb4a15b0a193ddf00a6e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.769984 17428 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:07.795181 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8510a331dab34f468a303fa5b0054421" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
W20240501 08:04:07.798065 17277 leader_election.cc:341] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:07.811272 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 946149c123e84f048c41fe4f86395165 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240501 08:04:07.812642 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 946149c123e84f048c41fe4f86395165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.817293 17428 tablet_bootstrap.cc:654] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:07.819072 17436 raft_consensus.cc:482] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:07.820278 17226 tablet_service.cc:1467] Processing CreateTablet for tablet f12aee8e217041f19aa8102e5fe1507f (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240501 08:04:07.824493 17294 tablet_service.cc:1467] Processing CreateTablet for tablet a8945347fd484e038a72f8580fc88472 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240501 08:04:07.825768 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8945347fd484e038a72f8580fc88472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.831087 17221 tablet_service.cc:1467] Processing CreateTablet for tablet a8945347fd484e038a72f8580fc88472 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240501 08:04:07.831324 17219 tablet_service.cc:1467] Processing CreateTablet for tablet 35d9ba955a5347ac9cc55e89b2b42e4e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240501 08:04:07.832469 17299 tablet_service.cc:1467] Processing CreateTablet for tablet f12aee8e217041f19aa8102e5fe1507f (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240501 08:04:07.833688 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12aee8e217041f19aa8102e5fe1507f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.843642 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12aee8e217041f19aa8102e5fe1507f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.846068 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8945347fd484e038a72f8580fc88472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.853883 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8c13df28b14b428733c4c25c234a6b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:07.858206 17302 tablet_service.cc:1467] Processing CreateTablet for tablet 946149c123e84f048c41fe4f86395165 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240501 08:04:07.859491 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 946149c123e84f048c41fe4f86395165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.823112 17436 raft_consensus.cc:504] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.862883 17436 leader_election.cc:288] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:07.868669 17225 tablet_service.cc:1467] Processing CreateTablet for tablet 7a64317c43cd4a2598c392879c3ff65a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240501 08:04:07.869992 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a64317c43cd4a2598c392879c3ff65a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.870157 17371 tablet_service.cc:1467] Processing CreateTablet for tablet f12aee8e217041f19aa8102e5fe1507f (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240501 08:04:07.870312 17372 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa513fcceeb4a15b0a193ddf00a6e62 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240501 08:04:07.870764 17367 tablet_service.cc:1467] Processing CreateTablet for tablet a8945347fd484e038a72f8580fc88472 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240501 08:04:07.871335 17371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12aee8e217041f19aa8102e5fe1507f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.871889 17367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8945347fd484e038a72f8580fc88472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.871774 17368 tablet_service.cc:1467] Processing CreateTablet for tablet 7a64317c43cd4a2598c392879c3ff65a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240501 08:04:07.868760 17369 tablet_service.cc:1467] Processing CreateTablet for tablet 946149c123e84f048c41fe4f86395165 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240501 08:04:07.873749 17368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a64317c43cd4a2598c392879c3ff65a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.874698 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 9343b763cf7743e88400ddcd42a6b931 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240501 08:04:07.874867 17376 tablet_service.cc:1467] Processing CreateTablet for tablet 35d9ba955a5347ac9cc55e89b2b42e4e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240501 08:04:07.883613 17376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d9ba955a5347ac9cc55e89b2b42e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.884248 17369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 946149c123e84f048c41fe4f86395165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.869066 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8510a331dab34f468a303fa5b0054421" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:07.875185 17298 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa513fcceeb4a15b0a193ddf00a6e62 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240501 08:04:07.876107 17222 tablet_service.cc:1467] Processing CreateTablet for tablet 867eb9d6d70b41ddb63cd0e79921d391 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240501 08:04:07.874609 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 867eb9d6d70b41ddb63cd0e79921d391 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240501 08:04:07.886294 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa513fcceeb4a15b0a193ddf00a6e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.877511 17297 tablet_service.cc:1467] Processing CreateTablet for tablet 7a64317c43cd4a2598c392879c3ff65a (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240501 08:04:07.885463 17426 tablet_bootstrap.cc:492] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:07.890631 17426 ts_tablet_manager.cc:1367] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.218s user 0.008s sys 0.006s
I20240501 08:04:07.890167 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867eb9d6d70b41ddb63cd0e79921d391. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.894855 17297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a64317c43cd4a2598c392879c3ff65a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.895911 17427 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:07.896378 17427 ts_tablet_manager.cc:1367] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.240s user 0.033s sys 0.020s
I20240501 08:04:07.874614 17219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d9ba955a5347ac9cc55e89b2b42e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.898123 17299 tablet_service.cc:1467] Processing CreateTablet for tablet 35d9ba955a5347ac9cc55e89b2b42e4e (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240501 08:04:07.898985 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa513fcceeb4a15b0a193ddf00a6e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.898933 17427 raft_consensus.cc:348] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:07.896375 17351 leader_election.cc:341] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:07.898098 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867eb9d6d70b41ddb63cd0e79921d391. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 08:04:07.903878 17278 leader_election.cc:341] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:07.904287 17278 leader_election.cc:302] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:07.904449 17296 tablet_service.cc:1467] Processing CreateTablet for tablet 867eb9d6d70b41ddb63cd0e79921d391 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240501 08:04:07.905290 17432 raft_consensus.cc:2726] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:07.900281 17426 raft_consensus.cc:348] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.914713 17301 tablet_service.cc:1467] Processing CreateTablet for tablet 9343b763cf7743e88400ddcd42a6b931 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240501 08:04:07.902297 17427 raft_consensus.cc:374] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.904273 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d9ba955a5347ac9cc55e89b2b42e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.921730 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9343b763cf7743e88400ddcd42a6b931. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.922281 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8c13df28b14b428733c4c25c234a6b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:07.922956 17312 raft_consensus.cc:2454] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:07.925737 17373 tablet_service.cc:1467] Processing CreateTablet for tablet 9343b763cf7743e88400ddcd42a6b931 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240501 08:04:07.927091 17373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9343b763cf7743e88400ddcd42a6b931. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.918541 17427 raft_consensus.cc:729] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.930043 17427 consensus_queue.cc:260] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.932101 17427 ts_tablet_manager.cc:1398] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.035s user 0.004s sys 0.000s
I20240501 08:04:07.932936 17427 tablet_bootstrap.cc:492] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:07.933466 17220 tablet_service.cc:1467] Processing CreateTablet for tablet 96c8eff63a7449018719d17f779939c5 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240501 08:04:07.937273 17295 tablet_service.cc:1467] Processing CreateTablet for tablet 96c8eff63a7449018719d17f779939c5 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240501 08:04:07.939513 17227 tablet_service.cc:1467] Processing CreateTablet for tablet e06988f9531b4744b9859548c8cc6530 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240501 08:04:07.941512 17301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9343b763cf7743e88400ddcd42a6b931. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.942505 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96c8eff63a7449018719d17f779939c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.945402 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867eb9d6d70b41ddb63cd0e79921d391. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.948335 17300 tablet_service.cc:1467] Processing CreateTablet for tablet e06988f9531b4744b9859548c8cc6530 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240501 08:04:07.951067 17293 tablet_service.cc:1467] Processing CreateTablet for tablet b3869c361a6d47679025e9c5e47b4706 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240501 08:04:07.952239 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3869c361a6d47679025e9c5e47b4706. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.917017 17426 raft_consensus.cc:374] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:07.954134 17426 raft_consensus.cc:729] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:07.954847 17426 consensus_queue.cc:260] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.966207 17433 raft_consensus.cc:482] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:07.966717 17433 raft_consensus.cc:504] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:07.973325 17220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96c8eff63a7449018719d17f779939c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.973577 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06988f9531b4744b9859548c8cc6530. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.975087 17227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06988f9531b4744b9859548c8cc6530. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.976683 17351 leader_election.cc:302] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:07.994446 17224 tablet_service.cc:1467] Processing CreateTablet for tablet b3869c361a6d47679025e9c5e47b4706 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240501 08:04:07.995776 17433 leader_election.cc:288] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.002060 17375 tablet_service.cc:1467] Processing CreateTablet for tablet e06988f9531b4744b9859548c8cc6530 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240501 08:04:08.003365 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06988f9531b4744b9859548c8cc6530. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.006251 17370 tablet_service.cc:1467] Processing CreateTablet for tablet b3869c361a6d47679025e9c5e47b4706 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240501 08:04:08.007505 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3869c361a6d47679025e9c5e47b4706. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:07.998311 17426 ts_tablet_manager.cc:1398] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.107s user 0.004s sys 0.004s
I20240501 08:04:07.997069 17367 tablet_service.cc:1467] Processing CreateTablet for tablet 96c8eff63a7449018719d17f779939c5 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240501 08:04:08.017940 17367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96c8eff63a7449018719d17f779939c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.018939 17226 tablet_service.cc:1467] Processing CreateTablet for tablet 78bfdb70077142128e0514f841189ca5 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240501 08:04:08.020105 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78bfdb70077142128e0514f841189ca5. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 08:04:08.021358 17351 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.130:33757, user_credentials={real_user=slave}} blocked reactor thread for 51339.5us
I20240501 08:04:08.022969 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f430c048c149c889744933e9c6a2ac" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
W20240501 08:04:08.024435 17351 leader_election.cc:341] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.027868 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 78bfdb70077142128e0514f841189ca5 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240501 08:04:08.029160 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78bfdb70077142128e0514f841189ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.032560 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3869c361a6d47679025e9c5e47b4706. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.009693 17433 raft_consensus.cc:2781] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.033557 17433 raft_consensus.cc:482] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.033924 17433 raft_consensus.cc:3037] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.036656 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f430c048c149c889744933e9c6a2ac" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.037919 17428 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.038401 17428 ts_tablet_manager.cc:1367] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.269s user 0.014s sys 0.000s
I20240501 08:04:08.040428 17298 tablet_service.cc:1467] Processing CreateTablet for tablet 78bfdb70077142128e0514f841189ca5 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20240501 08:04:08.041780 17351 leader_election.cc:341] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.039413 17372 tablet_service.cc:1467] Processing CreateTablet for tablet a0fe90f0956c48d184cd48f76c7e2a64 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240501 08:04:08.042224 17351 leader_election.cc:302] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.043009 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fe90f0956c48d184cd48f76c7e2a64. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.042543 17219 tablet_service.cc:1467] Processing CreateTablet for tablet d3abec6d380a4a5fa72fd34fe37bbd92 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240501 08:04:08.043418 17450 raft_consensus.cc:2726] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.043996 17219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3abec6d380a4a5fa72fd34fe37bbd92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.038334 17223 tablet_service.cc:1467] Processing CreateTablet for tablet a0fe90f0956c48d184cd48f76c7e2a64 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240501 08:04:08.047302 17426 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.047227 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fe90f0956c48d184cd48f76c7e2a64. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.048254 17222 tablet_service.cc:1467] Processing CreateTablet for tablet e93329da82834bd19fd547b7a5abdadb (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240501 08:04:08.046042 17294 tablet_service.cc:1467] Processing CreateTablet for tablet a0fe90f0956c48d184cd48f76c7e2a64 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240501 08:04:08.050631 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fe90f0956c48d184cd48f76c7e2a64. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.062258 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.063542 17298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78bfdb70077142128e0514f841189ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.063982 17295 tablet_service.cc:1467] Processing CreateTablet for tablet e93329da82834bd19fd547b7a5abdadb (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240501 08:04:08.044077 17428 raft_consensus.cc:348] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.065515 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93329da82834bd19fd547b7a5abdadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.059043 17302 tablet_service.cc:1467] Processing CreateTablet for tablet d3abec6d380a4a5fa72fd34fe37bbd92 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240501 08:04:08.131897 17428 raft_consensus.cc:374] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.132302 17428 raft_consensus.cc:729] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.132746 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3abec6d380a4a5fa72fd34fe37bbd92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.133041 17428 consensus_queue.cc:260] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.068838 17375 tablet_service.cc:1467] Processing CreateTablet for tablet d3abec6d380a4a5fa72fd34fe37bbd92 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240501 08:04:08.134897 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3abec6d380a4a5fa72fd34fe37bbd92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.074028 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.056111 17427 tablet_bootstrap.cc:654] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:08.137485 17203 leader_election.cc:341] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.056385 17433 raft_consensus.cc:504] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.141350 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8c13df28b14b428733c4c25c234a6b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.142282 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93329da82834bd19fd547b7a5abdadb. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 08:04:08.142710 17351 leader_election.cc:341] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.183173 17426 tablet_bootstrap.cc:654] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:08.185489 17202 leader_election.cc:341] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.186980 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8c13df28b14b428733c4c25c234a6b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.187472 17433 leader_election.cc:288] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.193423 17428 ts_tablet_manager.cc:1398] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.155s user 0.013s sys 0.000s
I20240501 08:04:08.194296 17428 tablet_bootstrap.cc:492] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
W20240501 08:04:08.201431 17122 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.168.131:43915, user_credentials={real_user=slave}} blocked reactor thread for 122271us
W20240501 08:04:08.202718 17121 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.168.130:33757, user_credentials={real_user=slave}} blocked reactor thread for 115751us
I20240501 08:04:08.204291 17312 raft_consensus.cc:3037] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.204932 17202 leader_election.cc:302] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:08.206382 17436 raft_consensus.cc:2726] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.209076 17370 tablet_service.cc:1467] Processing CreateTablet for tablet e93329da82834bd19fd547b7a5abdadb (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240501 08:04:08.210531 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93329da82834bd19fd547b7a5abdadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.230592 17376 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd19ca98cfd4974ad3fca19eb088cc0 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240501 08:04:08.231946 17376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd19ca98cfd4974ad3fca19eb088cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.235807 17428 tablet_bootstrap.cc:654] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.239316 17369 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee19995987e4749b8c949a60e1ae1e9 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240501 08:04:08.240527 17369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee19995987e4749b8c949a60e1ae1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.243752 17432 raft_consensus.cc:482] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.244208 17432 raft_consensus.cc:504] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.247356 17432 leader_election.cc:288] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.251374 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
W20240501 08:04:08.252728 17278 leader_election.cc:341] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.254232 17312 raft_consensus.cc:2454] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:08.255270 17351 leader_election.cc:302] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.256068 17433 raft_consensus.cc:2781] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.256601 17433 raft_consensus.cc:686] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:08.257522 17433 consensus_queue.cc:237] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.256811 17426 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.258845 17426 ts_tablet_manager.cc:1367] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.212s user 0.008s sys 0.007s
I20240501 08:04:08.260924 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.268569 17226 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd19ca98cfd4974ad3fca19eb088cc0 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240501 08:04:08.270897 17225 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee19995987e4749b8c949a60e1ae1e9 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240501 08:04:08.272058 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee19995987e4749b8c949a60e1ae1e9. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 08:04:08.272557 17277 leader_election.cc:341] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.272949 17277 leader_election.cc:302] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.274730 17432 raft_consensus.cc:2726] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.274888 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd19ca98cfd4974ad3fca19eb088cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.277942 17297 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee19995987e4749b8c949a60e1ae1e9 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240501 08:04:08.279079 17297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee19995987e4749b8c949a60e1ae1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.290166 17427 tablet_bootstrap.cc:492] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.290694 17427 ts_tablet_manager.cc:1367] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.358s user 0.038s sys 0.009s
I20240501 08:04:08.290319 17428 tablet_bootstrap.cc:492] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.291687 17428 ts_tablet_manager.cc:1367] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.098s user 0.013s sys 0.033s
I20240501 08:04:08.263624 17426 raft_consensus.cc:348] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.294025 17426 raft_consensus.cc:374] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.293668 17427 raft_consensus.cc:348] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.294657 17427 raft_consensus.cc:374] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.294366 17426 raft_consensus.cc:729] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.295035 17427 raft_consensus.cc:729] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.296005 17426 consensus_queue.cc:260] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.294342 17428 raft_consensus.cc:348] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.297021 17428 raft_consensus.cc:374] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.304194 17428 raft_consensus.cc:729] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.296006 17427 consensus_queue.cc:260] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.296308 17295 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd19ca98cfd4974ad3fca19eb088cc0 (DEFAULT_TABLE table=test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240501 08:04:08.317225 17427 ts_tablet_manager.cc:1398] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20240501 08:04:08.305076 17428 consensus_queue.cc:260] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.318806 17450 raft_consensus.cc:482] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.319381 17450 raft_consensus.cc:504] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.323612 17450 leader_election.cc:288] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.325874 17295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd19ca98cfd4974ad3fca19eb088cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:08.326460 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41080472c7646c68dd171e816059375" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:08.327473 17351 leader_election.cc:341] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.329676 17427 tablet_bootstrap.cc:492] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.329573 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41080472c7646c68dd171e816059375" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.330565 17428 ts_tablet_manager.cc:1398] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.039s user 0.009s sys 0.008s
I20240501 08:04:08.331532 17428 tablet_bootstrap.cc:492] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.331977 17134 catalog_manager.cc:5484] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
W20240501 08:04:08.333494 17351 leader_election.cc:341] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.333999 17426 ts_tablet_manager.cc:1398] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.075s user 0.006s sys 0.000s
I20240501 08:04:08.333922 17351 leader_election.cc:302] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.337044 17426 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.337646 17428 tablet_bootstrap.cc:654] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.343868 17455 raft_consensus.cc:2726] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.362725 17426 tablet_bootstrap.cc:654] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.367991 17427 tablet_bootstrap.cc:654] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.368008 17428 tablet_bootstrap.cc:492] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.368809 17428 ts_tablet_manager.cc:1367] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.037s user 0.025s sys 0.011s
I20240501 08:04:08.371814 17428 raft_consensus.cc:348] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.372602 17428 raft_consensus.cc:374] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.372925 17428 raft_consensus.cc:729] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.373633 17426 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.374119 17426 ts_tablet_manager.cc:1367] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.037s user 0.017s sys 0.003s
I20240501 08:04:08.373776 17428 consensus_queue.cc:260] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.376346 17428 ts_tablet_manager.cc:1398] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20240501 08:04:08.377338 17428 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.376978 17426 raft_consensus.cc:348] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.377733 17426 raft_consensus.cc:374] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.378005 17436 raft_consensus.cc:482] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.378032 17426 raft_consensus.cc:729] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.378659 17436 raft_consensus.cc:504] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.379043 17426 consensus_queue.cc:260] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.380662 17436 leader_election.cc:288] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.382206 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2250a60c258544668fa164fcf11634e4" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.382525 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2250a60c258544668fa164fcf11634e4" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.382804 17312 raft_consensus.cc:2454] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:08.383626 17428 tablet_bootstrap.cc:654] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:08.383867 17203 leader_election.cc:341] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.384397 17202 leader_election.cc:302] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:08.384814 17426 ts_tablet_manager.cc:1398] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240501 08:04:08.385251 17427 tablet_bootstrap.cc:492] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.385394 17436 raft_consensus.cc:2781] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.385736 17436 raft_consensus.cc:482] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.385818 17426 tablet_bootstrap.cc:492] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.386144 17427 ts_tablet_manager.cc:1367] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.057s user 0.013s sys 0.010s
I20240501 08:04:08.386102 17436 raft_consensus.cc:3037] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.388919 17427 raft_consensus.cc:348] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.389845 17427 raft_consensus.cc:374] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.390168 17427 raft_consensus.cc:729] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.390849 17427 consensus_queue.cc:260] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.393476 17426 tablet_bootstrap.cc:654] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.393174 17436 raft_consensus.cc:504] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.394840 17428 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.395510 17428 ts_tablet_manager.cc:1367] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
W20240501 08:04:08.400503 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:08.408079 17426 tablet_bootstrap.cc:492] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.408553 17426 ts_tablet_manager.cc:1367] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.000s
I20240501 08:04:08.410936 17436 leader_election.cc:288] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.411167 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2250a60c258544668fa164fcf11634e4" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.411613 17436 raft_consensus.cc:482] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.411237 17426 raft_consensus.cc:348] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.412074 17426 raft_consensus.cc:374] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.412398 17426 raft_consensus.cc:729] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.412024 17436 raft_consensus.cc:504] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.414381 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2250a60c258544668fa164fcf11634e4" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.398167 17428 raft_consensus.cc:348] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.414968 17428 raft_consensus.cc:374] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.414975 17312 raft_consensus.cc:3037] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.415401 17428 raft_consensus.cc:729] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.416168 17428 consensus_queue.cc:260] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.417055 17386 raft_consensus.cc:3037] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.418398 17428 ts_tablet_manager.cc:1398] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20240501 08:04:08.419128 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8c13df28b14b428733c4c25c234a6b" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.419925 17311 raft_consensus.cc:2379] T 4c8c13df28b14b428733c4c25c234a6b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:08.413125 17426 consensus_queue.cc:260] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.419346 17428 tablet_bootstrap.cc:492] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.421963 17426 ts_tablet_manager.cc:1398] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20240501 08:04:08.422883 17426 tablet_bootstrap.cc:492] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.423534 17386 raft_consensus.cc:2454] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:08.426149 17436 leader_election.cc:288] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.426487 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8c13df28b14b428733c4c25c234a6b" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.428413 17203 leader_election.cc:302] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:08.428633 17427 ts_tablet_manager.cc:1398] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.042s user 0.018s sys 0.013s
I20240501 08:04:08.429147 17428 tablet_bootstrap.cc:654] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.429764 17427 tablet_bootstrap.cc:492] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.430794 17436 raft_consensus.cc:3037] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.433854 17426 tablet_bootstrap.cc:654] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.428755 17312 raft_consensus.cc:2454] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:08.440639 17203 leader_election.cc:302] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.442054 17436 raft_consensus.cc:2726] T 4c8c13df28b14b428733c4c25c234a6b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.442366 17464 raft_consensus.cc:2781] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.442904 17464 raft_consensus.cc:686] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:08.443735 17464 consensus_queue.cc:237] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.445022 17428 tablet_bootstrap.cc:492] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.445530 17428 ts_tablet_manager.cc:1367] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.007s
I20240501 08:04:08.448235 17427 tablet_bootstrap.cc:654] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.448235 17428 raft_consensus.cc:348] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.449223 17428 raft_consensus.cc:374] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.449553 17428 raft_consensus.cc:729] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.450309 17428 consensus_queue.cc:260] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.452607 17428 ts_tablet_manager.cc:1398] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240501 08:04:08.453588 17428 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.459081 17428 tablet_bootstrap.cc:654] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.459246 17427 tablet_bootstrap.cc:492] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.459700 17427 ts_tablet_manager.cc:1367] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.003s
I20240501 08:04:08.461921 17427 raft_consensus.cc:348] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.462442 17427 raft_consensus.cc:374] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.462682 17427 raft_consensus.cc:729] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.463263 17427 consensus_queue.cc:260] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.464536 17455 raft_consensus.cc:482] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.465308 17427 ts_tablet_manager.cc:1398] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20240501 08:04:08.464983 17455 raft_consensus.cc:504] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.466349 17427 tablet_bootstrap.cc:492] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.467271 17455 leader_election.cc:288] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.468319 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e00624b7db1409890298f5a6d88a575" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:08.469861 17351 leader_election.cc:341] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.471128 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e00624b7db1409890298f5a6d88a575" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
W20240501 08:04:08.472440 17351 leader_election.cc:341] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.472635 17427 tablet_bootstrap.cc:654] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.472865 17426 tablet_bootstrap.cc:492] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.473462 17426 ts_tablet_manager.cc:1367] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.021s
I20240501 08:04:08.474400 17351 leader_election.cc:302] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.475167 17455 raft_consensus.cc:2726] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.476531 17426 raft_consensus.cc:348] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.477384 17426 raft_consensus.cc:374] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.477715 17426 raft_consensus.cc:729] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.478592 17426 consensus_queue.cc:260] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.481010 17133 catalog_manager.cc:5484] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:08.483713 17426 ts_tablet_manager.cc:1398] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.004s sys 0.001s
I20240501 08:04:08.484651 17426 tablet_bootstrap.cc:492] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.486732 17428 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.487238 17428 ts_tablet_manager.cc:1367] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.013s
I20240501 08:04:08.488282 17427 tablet_bootstrap.cc:492] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.488785 17427 ts_tablet_manager.cc:1367] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.013s
I20240501 08:04:08.490010 17426 tablet_bootstrap.cc:654] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.489727 17428 raft_consensus.cc:348] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.490412 17428 raft_consensus.cc:374] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.490680 17428 raft_consensus.cc:729] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.491303 17428 consensus_queue.cc:260] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.491598 17427 raft_consensus.cc:348] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.496428 17428 ts_tablet_manager.cc:1398] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20240501 08:04:08.497213 17428 tablet_bootstrap.cc:492] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.497926 17427 raft_consensus.cc:374] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.498345 17427 raft_consensus.cc:729] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.499164 17427 consensus_queue.cc:260] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.501811 17428 tablet_bootstrap.cc:654] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.501847 17427 ts_tablet_manager.cc:1398] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240501 08:04:08.503018 17427 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.508486 17427 tablet_bootstrap.cc:654] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.512200 17426 tablet_bootstrap.cc:492] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.512706 17426 ts_tablet_manager.cc:1367] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.009s
I20240501 08:04:08.513857 17428 tablet_bootstrap.cc:492] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.514259 17428 ts_tablet_manager.cc:1367] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20240501 08:04:08.515470 17426 raft_consensus.cc:348] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.516413 17426 raft_consensus.cc:374] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.516358 17428 raft_consensus.cc:348] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.516853 17426 raft_consensus.cc:729] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.517119 17428 raft_consensus.cc:374] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.517449 17428 raft_consensus.cc:729] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.517832 17426 consensus_queue.cc:260] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.518034 17428 consensus_queue.cc:260] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.520694 17427 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.520777 17428 ts_tablet_manager.cc:1398] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:08.521183 17427 ts_tablet_manager.cc:1367] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.001s
I20240501 08:04:08.521698 17428 tablet_bootstrap.cc:492] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.523658 17427 raft_consensus.cc:348] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.524360 17427 raft_consensus.cc:374] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.524623 17427 raft_consensus.cc:729] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.525426 17426 ts_tablet_manager.cc:1398] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.012s user 0.003s sys 0.003s
I20240501 08:04:08.525337 17427 consensus_queue.cc:260] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.526806 17428 tablet_bootstrap.cc:654] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.527590 17427 ts_tablet_manager.cc:1398] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:08.528699 17426 tablet_bootstrap.cc:492] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.528767 17427 tablet_bootstrap.cc:492] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.533840 17426 tablet_bootstrap.cc:654] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.534137 17427 tablet_bootstrap.cc:654] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:08.541671 17263 tablet.cc:2369] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:04:08.550020 17428 tablet_bootstrap.cc:492] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.550665 17428 ts_tablet_manager.cc:1367] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.029s user 0.017s sys 0.011s
I20240501 08:04:08.551856 17427 tablet_bootstrap.cc:492] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.551987 17426 tablet_bootstrap.cc:492] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.552235 17427 ts_tablet_manager.cc:1367] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.011s
I20240501 08:04:08.552407 17426 ts_tablet_manager.cc:1367] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.003s
I20240501 08:04:08.552979 17428 raft_consensus.cc:348] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.553678 17428 raft_consensus.cc:374] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.553936 17428 raft_consensus.cc:729] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.554158 17427 raft_consensus.cc:348] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.554706 17427 raft_consensus.cc:374] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.554529 17428 consensus_queue.cc:260] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.555090 17427 raft_consensus.cc:729] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.554697 17426 raft_consensus.cc:348] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.555509 17426 raft_consensus.cc:374] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.555760 17426 raft_consensus.cc:729] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.555768 17427 consensus_queue.cc:260] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.556551 17428 ts_tablet_manager.cc:1398] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240501 08:04:08.556460 17426 consensus_queue.cc:260] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.557464 17428 tablet_bootstrap.cc:492] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.558413 17426 ts_tablet_manager.cc:1398] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:04:08.559201 17426 tablet_bootstrap.cc:492] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.562626 17428 tablet_bootstrap.cc:654] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.563832 17426 tablet_bootstrap.cc:654] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.565191 17427 ts_tablet_manager.cc:1398] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20240501 08:04:08.569046 17427 tablet_bootstrap.cc:492] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.574549 17464 raft_consensus.cc:482] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.575014 17464 raft_consensus.cc:504] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.576929 17464 leader_election.cc:288] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.581017 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f430c048c149c889744933e9c6a2ac" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.578996 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f430c048c149c889744933e9c6a2ac" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.584429 17386 raft_consensus.cc:2454] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:08.584517 17428 tablet_bootstrap.cc:492] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.583886 17312 raft_consensus.cc:2454] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:08.584980 17428 ts_tablet_manager.cc:1367] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.003s
I20240501 08:04:08.585464 17426 tablet_bootstrap.cc:492] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.585515 17203 leader_election.cc:302] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.585937 17426 ts_tablet_manager.cc:1367] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.016s
I20240501 08:04:08.583884 17427 tablet_bootstrap.cc:654] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.586400 17464 raft_consensus.cc:2781] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.586793 17464 raft_consensus.cc:482] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.587141 17464 raft_consensus.cc:3037] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.587395 17428 raft_consensus.cc:348] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.588057 17428 raft_consensus.cc:374] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.588294 17428 raft_consensus.cc:729] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.588590 17426 raft_consensus.cc:348] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.588863 17428 consensus_queue.cc:260] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.589274 17426 raft_consensus.cc:374] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.589587 17426 raft_consensus.cc:729] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.590159 17426 consensus_queue.cc:260] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.591981 17426 ts_tablet_manager.cc:1398] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:08.595793 17426 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.597640 17428 ts_tablet_manager.cc:1398] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20240501 08:04:08.598997 17436 raft_consensus.cc:482] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.599478 17436 raft_consensus.cc:504] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.600873 17426 tablet_bootstrap.cc:654] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.601413 17436 leader_election.cc:288] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.601760 17428 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.602726 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e141ff8f064eb480d6e2174c541e42" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.603435 17386 raft_consensus.cc:2454] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:08.604446 17203 leader_election.cc:302] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.605309 17436 raft_consensus.cc:2781] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.605662 17436 raft_consensus.cc:482] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.605970 17436 raft_consensus.cc:3037] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.607122 17428 tablet_bootstrap.cc:654] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.609683 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e141ff8f064eb480d6e2174c541e42" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.614979 17427 tablet_bootstrap.cc:492] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
W20240501 08:04:08.614914 17202 leader_election.cc:341] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.615499 17427 ts_tablet_manager.cc:1367] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.014s
I20240501 08:04:08.594663 17464 raft_consensus.cc:504] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.613034 17436 raft_consensus.cc:504] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.618834 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f430c048c149c889744933e9c6a2ac" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.619544 17312 raft_consensus.cc:3037] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.620172 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e141ff8f064eb480d6e2174c541e42" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
W20240501 08:04:08.621456 17202 leader_election.cc:341] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.623375 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e141ff8f064eb480d6e2174c541e42" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.623955 17386 raft_consensus.cc:3037] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.627163 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f430c048c149c889744933e9c6a2ac" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.627753 17385 raft_consensus.cc:3037] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.630311 17386 raft_consensus.cc:2454] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:08.631424 17203 leader_election.cc:302] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:08.632431 17466 raft_consensus.cc:2781] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.634763 17385 raft_consensus.cc:2454] T 68f430c048c149c889744933e9c6a2ac P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:08.636046 17203 leader_election.cc:302] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.638602 17436 leader_election.cc:288] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.617810 17427 raft_consensus.cc:348] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.640955 17427 raft_consensus.cc:374] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.641255 17427 raft_consensus.cc:729] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.640611 17464 leader_election.cc:288] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.642062 17464 raft_consensus.cc:2781] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.644073 17464 raft_consensus.cc:686] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:08.645642 17464 consensus_queue.cc:237] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.642973 17312 raft_consensus.cc:2454] T 68f430c048c149c889744933e9c6a2ac P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:08.646318 17466 raft_consensus.cc:686] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:08.648550 17466 consensus_queue.cc:237] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.649514 17426 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.650149 17426 ts_tablet_manager.cc:1367] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.055s user 0.021s sys 0.031s
I20240501 08:04:08.643923 17427 consensus_queue.cc:260] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.652639 17432 raft_consensus.cc:482] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.653198 17428 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.652791 17426 raft_consensus.cc:348] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.653103 17432 raft_consensus.cc:504] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.653563 17427 ts_tablet_manager.cc:1398] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20240501 08:04:08.653681 17426 raft_consensus.cc:374] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.653875 17428 ts_tablet_manager.cc:1367] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.052s user 0.028s sys 0.023s
I20240501 08:04:08.654009 17426 raft_consensus.cc:729] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.654670 17427 tablet_bootstrap.cc:492] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.654968 17432 leader_election.cc:288] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.654726 17426 consensus_queue.cc:260] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.655643 17455 raft_consensus.cc:482] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.656069 17455 raft_consensus.cc:504] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.656723 17426 ts_tablet_manager.cc:1398] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240501 08:04:08.657709 17432 raft_consensus.cc:482] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.657891 17455 leader_election.cc:288] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:08.658133 17432 raft_consensus.cc:504] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.659657 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8510a331dab34f468a303fa5b0054421" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.660245 17311 raft_consensus.cc:2454] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:08.660399 17427 tablet_bootstrap.cc:654] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.660467 17426 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.660830 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.661556 17351 leader_election.cc:302] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:08.656788 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41080472c7646c68dd171e816059375" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.662386 17385 raft_consensus.cc:2454] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
W20240501 08:04:08.662501 17278 leader_election.cc:341] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.663400 17278 leader_election.cc:302] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:08.664651 17432 leader_election.cc:288] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.664870 17455 raft_consensus.cc:2781] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.665139 17432 raft_consensus.cc:2781] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.665205 17455 raft_consensus.cc:482] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.665542 17432 raft_consensus.cc:482] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.665620 17455 raft_consensus.cc:3037] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.666153 17426 tablet_bootstrap.cc:654] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.667603 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8510a331dab34f468a303fa5b0054421" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.668213 17237 raft_consensus.cc:2454] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:08.665856 17432 raft_consensus.cc:3037] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.669875 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41080472c7646c68dd171e816059375" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.670471 17237 raft_consensus.cc:2454] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:08.677739 17426 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.678218 17426 ts_tablet_manager.cc:1367] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
I20240501 08:04:08.679309 17428 raft_consensus.cc:348] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.680750 17428 raft_consensus.cc:374] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.680967 17426 raft_consensus.cc:348] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.684243 17426 raft_consensus.cc:374] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.683354 17432 raft_consensus.cc:504] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.684003 17428 raft_consensus.cc:729] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.684530 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.680328 17455 raft_consensus.cc:504] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.684540 17426 raft_consensus.cc:729] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.685952 17432 leader_election.cc:288] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.686923 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41080472c7646c68dd171e816059375" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.685959 17426 consensus_queue.cc:260] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.687574 17385 raft_consensus.cc:3037] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.688985 17426 ts_tablet_manager.cc:1398] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240501 08:04:08.689831 17426 tablet_bootstrap.cc:492] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
W20240501 08:04:08.687017 17277 leader_election.cc:341] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.693199 17277 leader_election.cc:302] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.693706 17385 raft_consensus.cc:2454] T b41080472c7646c68dd171e816059375 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:08.694214 17432 raft_consensus.cc:2726] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.694715 17426 tablet_bootstrap.cc:654] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.694739 17278 leader_election.cc:302] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:08.695680 17432 raft_consensus.cc:2781] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.696139 17432 raft_consensus.cc:686] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:08.696333 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41080472c7646c68dd171e816059375" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.696568 17428 consensus_queue.cc:260] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.696985 17237 raft_consensus.cc:3037] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.699399 17488 raft_consensus.cc:482] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.699298 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8510a331dab34f468a303fa5b0054421" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.700007 17312 raft_consensus.cc:3037] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.699916 17488 raft_consensus.cc:504] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.701979 17488 leader_election.cc:288] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.703459 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fd46911e2c4ac8a341f1d910bc978e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.704120 17386 raft_consensus.cc:2454] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:08.705318 17278 leader_election.cc:302] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:08.706143 17488 raft_consensus.cc:2781] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.706473 17488 raft_consensus.cc:482] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.706666 17312 raft_consensus.cc:2454] T 8510a331dab34f468a303fa5b0054421 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:08.706810 17488 raft_consensus.cc:3037] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.708787 17428 ts_tablet_manager.cc:1398] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.054s user 0.005s sys 0.000s
I20240501 08:04:08.709719 17428 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.710391 17433 raft_consensus.cc:482] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.710858 17433 raft_consensus.cc:504] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.707477 17455 leader_election.cc:288] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:08.713670 17432 consensus_queue.cc:237] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.714139 17433 leader_election.cc:288] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.715051 17488 raft_consensus.cc:504] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.726290 17488 leader_election.cc:288] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.725409 17236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fd46911e2c4ac8a341f1d910bc978e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.736614 17428 tablet_bootstrap.cc:654] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.737752 17236 raft_consensus.cc:2454] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:08.728361 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fd46911e2c4ac8a341f1d910bc978e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.742285 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fd46911e2c4ac8a341f1d910bc978e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.743342 17234 raft_consensus.cc:3037] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.737900 17351 leader_election.cc:302] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:08.744637 17385 raft_consensus.cc:3037] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.743849 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8510a331dab34f468a303fa5b0054421" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.746806 17235 raft_consensus.cc:3037] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.745496 17433 raft_consensus.cc:2781] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.748303 17433 raft_consensus.cc:686] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:08.748950 17433 consensus_queue.cc:237] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } }
I20240501 08:04:08.751281 17237 raft_consensus.cc:2454] T b41080472c7646c68dd171e816059375 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:08.754185 17236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.756661 17133 catalog_manager.cc:5484] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:08.758311 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.758913 17312 raft_consensus.cc:2454] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
W20240501 08:04:08.759442 17351 leader_election.cc:341] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.759487 17135 catalog_manager.cc:5484] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:08.761407 17427 tablet_bootstrap.cc:492] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.762331 17427 ts_tablet_manager.cc:1367] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.108s user 0.018s sys 0.019s
I20240501 08:04:08.764935 17235 raft_consensus.cc:2454] T 8510a331dab34f468a303fa5b0054421 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:08.765439 17351 leader_election.cc:302] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.767163 17455 raft_consensus.cc:2781] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.767530 17455 raft_consensus.cc:482] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.767844 17455 raft_consensus.cc:3037] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.771679 17234 raft_consensus.cc:2454] T f3fd46911e2c4ac8a341f1d910bc978e P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:08.764739 17427 raft_consensus.cc:348] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.773499 17427 raft_consensus.cc:374] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.773768 17427 raft_consensus.cc:729] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.773674 17277 leader_election.cc:302] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.775056 17455 raft_consensus.cc:504] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.775261 17450 consensus_queue.cc:1035] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:08.776118 17428 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.776542 17428 ts_tablet_manager.cc:1367] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.067s user 0.006s sys 0.011s
I20240501 08:04:08.776979 17426 tablet_bootstrap.cc:492] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.778110 17426 ts_tablet_manager.cc:1367] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.088s user 0.033s sys 0.021s
I20240501 08:04:08.779899 17432 raft_consensus.cc:2781] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.779853 17427 consensus_queue.cc:260] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.780700 17426 raft_consensus.cc:348] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.781198 17432 raft_consensus.cc:686] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:08.781445 17426 raft_consensus.cc:374] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.781893 17426 raft_consensus.cc:729] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.782644 17426 consensus_queue.cc:260] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.767891 17385 raft_consensus.cc:2454] T f3fd46911e2c4ac8a341f1d910bc978e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:08.784639 17427 ts_tablet_manager.cc:1398] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.022s user 0.004s sys 0.001s
I20240501 08:04:08.785519 17427 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:08.789083 17455 leader_election.cc:288] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.779412 17428 raft_consensus.cc:348] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.789731 17428 raft_consensus.cc:374] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.790009 17428 raft_consensus.cc:729] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.789517 17134 catalog_manager.cc:5484] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:08.790613 17428 consensus_queue.cc:260] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.787196 17133 catalog_manager.cc:5484] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:08.792563 17428 ts_tablet_manager.cc:1398] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.016s user 0.002s sys 0.002s
I20240501 08:04:08.793380 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
W20240501 08:04:08.798625 17351 leader_election.cc:341] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.799120 17427 tablet_bootstrap.cc:654] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.799067 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.799721 17311 raft_consensus.cc:3037] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.800917 17428 tablet_bootstrap.cc:492] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.802269 17432 consensus_queue.cc:237] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.808534 17311 raft_consensus.cc:2454] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:08.817554 17351 leader_election.cc:302] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.818401 17455 raft_consensus.cc:2781] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.819156 17455 raft_consensus.cc:686] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:08.819975 17455 consensus_queue.cc:237] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.827651 17433 raft_consensus.cc:482] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.829771 17426 ts_tablet_manager.cc:1398] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.051s user 0.019s sys 0.025s
I20240501 08:04:08.828189 17433 raft_consensus.cc:504] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.832484 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1189be88e7b341918c1fa148b2d77030" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.833199 17433 leader_election.cc:288] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
W20240501 08:04:08.837788 17351 leader_election.cc:341] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.842350 17426 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:08.845983 17433 raft_consensus.cc:482] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.846583 17433 raft_consensus.cc:504] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.853029 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1189be88e7b341918c1fa148b2d77030" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.853781 17312 raft_consensus.cc:2454] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:08.854243 17488 raft_consensus.cc:482] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.854694 17488 raft_consensus.cc:504] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.855072 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.854694 17466 raft_consensus.cc:482] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.855116 17351 leader_election.cc:302] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.855772 17234 raft_consensus.cc:2454] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:08.855815 17466 raft_consensus.cc:504] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.857847 17428 tablet_bootstrap.cc:654] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.863664 17488 leader_election.cc:288] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.865137 17466 leader_election.cc:288] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.866030 17351 leader_election.cc:302] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.866513 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.874166 17234 raft_consensus.cc:2454] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:08.872887 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e00624b7db1409890298f5a6d88a575" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.876219 17385 raft_consensus.cc:2454] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:08.877952 17277 leader_election.cc:302] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.878751 17427 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.879355 17427 ts_tablet_manager.cc:1367] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.094s user 0.008s sys 0.003s
I20240501 08:04:08.877835 17503 raft_consensus.cc:2781] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.890296 17495 raft_consensus.cc:2781] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.882678 17427 raft_consensus.cc:348] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.890813 17495 raft_consensus.cc:482] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.890941 17427 raft_consensus.cc:374] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.891455 17427 raft_consensus.cc:729] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.880841 17203 leader_election.cc:302] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:08.892083 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e00624b7db1409890298f5a6d88a575" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.892918 17466 raft_consensus.cc:2781] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.893286 17466 raft_consensus.cc:482] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.879079 17433 leader_election.cc:288] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.892508 17427 consensus_queue.cc:260] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.877379 17133 catalog_manager.cc:5484] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:08.893637 17466 raft_consensus.cc:3037] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.881711 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:08.895579 17503 raft_consensus.cc:482] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.892685 17312 raft_consensus.cc:2454] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:08.891225 17495 raft_consensus.cc:3037] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.898200 17427 ts_tablet_manager.cc:1398] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.018s user 0.011s sys 0.000s
I20240501 08:04:08.895385 17385 raft_consensus.cc:2454] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:08.897486 17503 raft_consensus.cc:3037] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.906033 17455 raft_consensus.cc:482] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.908231 17433 raft_consensus.cc:2781] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.915979 17495 raft_consensus.cc:504] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.908399 17427 tablet_bootstrap.cc:492] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
W20240501 08:04:08.921555 17351 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.129:44541, user_credentials={real_user=slave}} blocked reactor thread for 63261.9us
I20240501 08:04:08.921147 17455 raft_consensus.cc:504] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.930517 17427 tablet_bootstrap.cc:654] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.932745 17433 raft_consensus.cc:482] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.932956 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:08.932660 17505 consensus_queue.cc:1035] T 4c8c13df28b14b428733c4c25c234a6b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:08.933943 17312 raft_consensus.cc:2379] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:08.919100 17466 raft_consensus.cc:504] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.935601 17433 raft_consensus.cc:3037] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.934389 17135 catalog_manager.cc:5484] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:08.939049 17426 tablet_bootstrap.cc:654] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:08.939183 17488 raft_consensus.cc:482] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.939628 17488 raft_consensus.cc:504] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.939944 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.940490 17234 raft_consensus.cc:3037] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.941545 17488 leader_election.cc:288] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.944723 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.945504 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
W20240501 08:04:08.947088 17278 leader_election.cc:341] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.943027 17495 leader_election.cc:288] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.944705 17455 leader_election.cc:288] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
W20240501 08:04:08.950296 17277 leader_election.cc:341] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.950747 17277 leader_election.cc:302] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:08.951496 17495 raft_consensus.cc:2726] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:08.947840 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.954800 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:08.955469 17235 raft_consensus.cc:2454] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:08.954910 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:08.957415 17351 leader_election.cc:341] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.958981 17351 leader_election.cc:302] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:08.959220 17433 raft_consensus.cc:504] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.959774 17455 raft_consensus.cc:2781] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:08.960376 17386 raft_consensus.cc:2379] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:08.960297 17455 raft_consensus.cc:482] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:08.960853 17455 raft_consensus.cc:3037] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.951777 17234 raft_consensus.cc:2454] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:08.964705 17277 leader_election.cc:302] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:08.967298 17495 raft_consensus.cc:2781] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:08.967775 17495 raft_consensus.cc:686] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:08.969872 17455 raft_consensus.cc:504] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.971832 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e00624b7db1409890298f5a6d88a575" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.972431 17311 raft_consensus.cc:3037] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.972690 17455 leader_election.cc:288] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.972363 17466 leader_election.cc:288] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:08.975184 17428 tablet_bootstrap.cc:492] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:08.975593 17428 ts_tablet_manager.cc:1367] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.182s user 0.022s sys 0.008s
I20240501 08:04:08.977142 17433 leader_election.cc:288] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:08.978236 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e00624b7db1409890298f5a6d88a575" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:08.978919 17386 raft_consensus.cc:3037] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.979257 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:08.979048 17495 consensus_queue.cc:237] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.979300 17428 raft_consensus.cc:348] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.980080 17428 raft_consensus.cc:374] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:08.980350 17428 raft_consensus.cc:729] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:08.980913 17428 consensus_queue.cc:260] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.978291 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.982920 17428 ts_tablet_manager.cc:1398] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20240501 08:04:08.984009 17234 raft_consensus.cc:3037] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:08.985522 17428 tablet_bootstrap.cc:492] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:08.983079 17466 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20240501 08:04:08.989367 17503 raft_consensus.cc:504] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:08.991755 17426 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:08.992242 17426 ts_tablet_manager.cc:1367] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.150s user 0.009s sys 0.002s
I20240501 08:04:08.995476 17427 tablet_bootstrap.cc:492] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:08.995268 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:08.996769 17235 raft_consensus.cc:2379] T 19b58febdb8147b699a76df8ce1cb923 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
W20240501 08:04:09.000536 17351 leader_election.cc:341] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:08.995474 17455 raft_consensus.cc:482] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:08.993808 17503 leader_election.cc:288] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.002935 17309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1189be88e7b341918c1fa148b2d77030" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:09.003564 17309 raft_consensus.cc:3037] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.000646 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19b58febdb8147b699a76df8ce1cb923" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:09.006991 17426 raft_consensus.cc:348] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.010211 17426 raft_consensus.cc:374] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.010548 17426 raft_consensus.cc:729] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.001097 17455 raft_consensus.cc:504] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.019409 17351 leader_election.cc:302] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.033339 17503 raft_consensus.cc:2726] T 19b58febdb8147b699a76df8ce1cb923 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.033494 17386 raft_consensus.cc:2454] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:09.034420 17427 ts_tablet_manager.cc:1367] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.128s user 0.039s sys 0.007s
I20240501 08:04:09.035780 17311 raft_consensus.cc:2454] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:09.012199 17234 raft_consensus.cc:2454] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:09.035265 17426 consensus_queue.cc:260] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.039549 17427 raft_consensus.cc:348] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.040560 17309 raft_consensus.cc:2454] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:09.041772 17426 ts_tablet_manager.cc:1398] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.049s user 0.012s sys 0.001s
I20240501 08:04:09.041541 17427 raft_consensus.cc:374] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.042372 17427 raft_consensus.cc:729] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.042868 17426 tablet_bootstrap.cc:492] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.042904 17427 consensus_queue.cc:260] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.044431 17202 leader_election.cc:302] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.044766 17427 ts_tablet_manager.cc:1398] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.009s user 0.003s sys 0.002s
I20240501 08:04:09.046145 17351 leader_election.cc:302] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:09.047168 17503 raft_consensus.cc:2781] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.047616 17503 raft_consensus.cc:686] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:09.047633 17464 raft_consensus.cc:2781] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.048740 17464 raft_consensus.cc:686] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:09.048584 17503 consensus_queue.cc:237] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.049515 17464 consensus_queue.cc:237] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.051290 17427 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.010916 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1189be88e7b341918c1fa148b2d77030" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:09.054350 17135 catalog_manager.cc:5484] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:09.052093 17495 raft_consensus.cc:482] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.055104 17235 raft_consensus.cc:3037] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.058707 17351 leader_election.cc:302] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:09.063931 17433 raft_consensus.cc:2781] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.064060 17466 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.064415 17433 raft_consensus.cc:686] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:09.065186 17433 consensus_queue.cc:237] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:09.070631 17202 consensus_peers.cc:485] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e -> Peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Couldn't send request to peer cdccadabd9154ea69e15405d75301fc1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:09.058669 17495 raft_consensus.cc:504] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.089850 17455 leader_election.cc:288] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.091949 17495 leader_election.cc:288] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.096055 17235 raft_consensus.cc:2454] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:09.097038 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e75ee3b0345ea8766285d1ef4e09e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.097965 17235 raft_consensus.cc:2454] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:09.098160 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e75ee3b0345ea8766285d1ef4e09e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.099704 17466 consensus_queue.cc:1035] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240501 08:04:09.102603 17277 leader_election.cc:302] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.103518 17488 raft_consensus.cc:2781] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.101894 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.104065 17488 raft_consensus.cc:482] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.104307 17312 raft_consensus.cc:2454] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:09.104527 17488 raft_consensus.cc:3037] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.099504 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.101441 17133 catalog_manager.cc:5484] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
W20240501 08:04:09.107288 17351 leader_election.cc:341] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.108238 17133 catalog_manager.cc:5484] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:09.109192 17134 catalog_manager.cc:5484] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:09.112223 17426 tablet_bootstrap.cc:654] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.116636 17503 raft_consensus.cc:482] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.117086 17503 raft_consensus.cc:504] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.117194 17488 raft_consensus.cc:504] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.117470 17351 leader_election.cc:302] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
W20240501 08:04:09.118271 17278 leader_election.cc:341] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.119567 17433 raft_consensus.cc:2781] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.119725 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.119946 17433 raft_consensus.cc:482] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.120280 17433 raft_consensus.cc:3037] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.117938 17495 raft_consensus.cc:482] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240501 08:04:09.121515 17351 leader_election.cc:341] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.118964 17503 leader_election.cc:288] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.125969 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.126675 17312 raft_consensus.cc:2454] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:09.127966 17351 leader_election.cc:302] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.128530 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e75ee3b0345ea8766285d1ef4e09e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:09.129153 17503 raft_consensus.cc:2781] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.129550 17235 raft_consensus.cc:3037] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.129631 17503 raft_consensus.cc:482] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.130226 17503 raft_consensus.cc:3037] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.135191 17495 raft_consensus.cc:504] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.126657 17433 raft_consensus.cc:504] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.136811 17427 tablet_bootstrap.cc:654] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.138713 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:09.140033 17235 raft_consensus.cc:2454] T 298e75ee3b0345ea8766285d1ef4e09e P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
W20240501 08:04:09.140261 17351 leader_election.cc:341] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.141005 17428 tablet_bootstrap.cc:654] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.142549 17277 leader_election.cc:302] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.143821 17432 raft_consensus.cc:2781] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.144663 17495 leader_election.cc:288] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.150213 17488 leader_election.cc:288] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.151849 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.156682 17432 raft_consensus.cc:686] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:09.157888 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e75ee3b0345ea8766285d1ef4e09e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:09.158219 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "068fca00dea44c6fbac5bc5365a66a4b" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.158926 17235 raft_consensus.cc:2379] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:09.160948 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
W20240501 08:04:09.161000 17278 leader_election.cc:341] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.161674 17312 raft_consensus.cc:3037] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.162586 17278 leader_election.cc:302] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.163859 17488 raft_consensus.cc:3037] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.151101 17433 leader_election.cc:288] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.168279 17432 consensus_queue.cc:237] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.168747 17466 consensus_queue.cc:1035] T 68f430c048c149c889744933e9c6a2ac P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.169858 17495 consensus_queue.cc:1035] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.171641 17312 raft_consensus.cc:2454] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:09.174183 17426 tablet_bootstrap.cc:492] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.174646 17426 ts_tablet_manager.cc:1367] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.132s user 0.010s sys 0.002s
I20240501 08:04:09.175209 17503 raft_consensus.cc:504] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.179073 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
W20240501 08:04:09.180650 17351 leader_election.cc:341] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.184216 17426 raft_consensus.cc:348] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.185561 17426 raft_consensus.cc:374] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.185882 17426 raft_consensus.cc:729] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.186529 17426 consensus_queue.cc:260] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.189309 17426 ts_tablet_manager.cc:1398] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20240501 08:04:09.190459 17426 tablet_bootstrap.cc:492] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.191673 17351 leader_election.cc:302] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.180732 17488 raft_consensus.cc:2726] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.195699 17427 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.198077 17427 ts_tablet_manager.cc:1367] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.147s user 0.011s sys 0.012s
I20240501 08:04:09.196215 17433 raft_consensus.cc:2781] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.200517 17433 raft_consensus.cc:686] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:09.205148 17426 tablet_bootstrap.cc:654] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.212966 17433 consensus_queue.cc:237] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.206104 17134 catalog_manager.cc:5484] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:09.219208 17427 raft_consensus.cc:348] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.219672 17427 raft_consensus.cc:374] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.219915 17427 raft_consensus.cc:729] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.220503 17427 consensus_queue.cc:260] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.223481 17503 leader_election.cc:288] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.227751 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:09.228309 17311 raft_consensus.cc:3037] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.232795 17311 raft_consensus.cc:2454] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:09.234616 17432 raft_consensus.cc:482] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.235177 17351 leader_election.cc:302] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.235086 17432 raft_consensus.cc:504] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.236121 17455 raft_consensus.cc:2781] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.236618 17455 raft_consensus.cc:686] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:09.236871 17432 leader_election.cc:288] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.237990 17427 ts_tablet_manager.cc:1398] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.040s user 0.000s sys 0.004s
I20240501 08:04:09.238809 17427 tablet_bootstrap.cc:492] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.243443 17428 tablet_bootstrap.cc:492] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.243180 17484 consensus_queue.cc:1035] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:09.243793 17428 ts_tablet_manager.cc:1367] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.258s user 0.016s sys 0.017s
I20240501 08:04:09.245759 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e141ff8f064eb480d6e2174c541e42" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.240836 17455 consensus_queue.cc:237] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.252475 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e141ff8f064eb480d6e2174c541e42" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.263397 17504 consensus_queue.cc:1035] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:09.270004 17495 consensus_queue.cc:1035] T b41080472c7646c68dd171e816059375 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.273698 17504 consensus_queue.cc:1035] T 8510a331dab34f468a303fa5b0054421 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.280771 17426 tablet_bootstrap.cc:492] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.281301 17426 ts_tablet_manager.cc:1367] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.091s user 0.032s sys 0.006s
I20240501 08:04:09.283802 17428 raft_consensus.cc:348] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.284787 17428 raft_consensus.cc:374] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.285050 17428 raft_consensus.cc:729] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.285660 17428 consensus_queue.cc:260] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.288074 17428 ts_tablet_manager.cc:1398] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.044s user 0.006s sys 0.001s
I20240501 08:04:09.288918 17428 tablet_bootstrap.cc:492] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.288509 17426 raft_consensus.cc:348] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.289288 17426 raft_consensus.cc:374] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.289563 17426 raft_consensus.cc:729] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.290215 17426 consensus_queue.cc:260] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:09.291529 17351 consensus_peers.cc:485] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Couldn't send request to peer fa9d3bc74a68481e9de647223cb0110e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:09.294026 17278 leader_election.cc:302] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.295140 17495 raft_consensus.cc:2726] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.309051 17426 ts_tablet_manager.cc:1398] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.027s user 0.007s sys 0.001s
I20240501 08:04:09.310532 17426 tablet_bootstrap.cc:492] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.341857 17426 tablet_bootstrap.cc:654] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.355098 17427 tablet_bootstrap.cc:654] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.339228 17433 consensus_queue.cc:1035] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20240501 08:04:09.373993 17135 catalog_manager.cc:5484] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:09.375788 17135 catalog_manager.cc:5484] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:09.385811 17495 consensus_queue.cc:1035] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.389187 17428 tablet_bootstrap.cc:654] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.405638 17488 raft_consensus.cc:482] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.406121 17488 raft_consensus.cc:504] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.410343 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ca37e5ca8844c68b9522e5d05bfc3e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.413686 17488 leader_election.cc:288] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:09.414747 17277 leader_election.cc:341] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.416326 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ca37e5ca8844c68b9522e5d05bfc3e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.418136 17427 tablet_bootstrap.cc:492] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.418555 17427 ts_tablet_manager.cc:1367] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.180s user 0.011s sys 0.010s
W20240501 08:04:09.418614 17278 leader_election.cc:341] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.419490 17278 leader_election.cc:302] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.420879 17427 raft_consensus.cc:348] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.421593 17427 raft_consensus.cc:374] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.421866 17427 raft_consensus.cc:729] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.422746 17495 raft_consensus.cc:2726] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.422508 17427 consensus_queue.cc:260] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.425442 17488 consensus_queue.cc:1035] T f3fd46911e2c4ac8a341f1d910bc978e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.427124 17427 ts_tablet_manager.cc:1398] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240501 08:04:09.428002 17427 tablet_bootstrap.cc:492] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.437870 17426 tablet_bootstrap.cc:492] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.438357 17426 ts_tablet_manager.cc:1367] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.128s user 0.017s sys 0.012s
W20240501 08:04:09.462643 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:09.467437 17426 raft_consensus.cc:348] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.470185 17426 raft_consensus.cc:374] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.470525 17426 raft_consensus.cc:729] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.471206 17426 consensus_queue.cc:260] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.472915 17426 ts_tablet_manager.cc:1398] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.034s user 0.003s sys 0.004s
I20240501 08:04:09.473778 17426 tablet_bootstrap.cc:492] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.481360 17503 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.486604 17428 tablet_bootstrap.cc:492] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.487067 17428 ts_tablet_manager.cc:1367] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.198s user 0.030s sys 0.037s
I20240501 08:04:09.487295 17504 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.487852 17427 tablet_bootstrap.cc:654] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.493793 17428 raft_consensus.cc:348] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.496928 17428 raft_consensus.cc:374] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.497562 17428 raft_consensus.cc:729] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.498497 17428 consensus_queue.cc:260] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.502203 17503 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.503536 17464 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.505746 17426 tablet_bootstrap.cc:654] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.511276 17428 ts_tablet_manager.cc:1398] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.024s user 0.001s sys 0.005s
I20240501 08:04:09.512177 17427 tablet_bootstrap.cc:492] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.512187 17428 tablet_bootstrap.cc:492] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.512697 17427 ts_tablet_manager.cc:1367] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.010s
I20240501 08:04:09.515015 17427 raft_consensus.cc:348] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.515517 17427 raft_consensus.cc:374] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.515714 17427 raft_consensus.cc:729] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.516233 17427 consensus_queue.cc:260] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.525493 17428 tablet_bootstrap.cc:654] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.525530 17427 ts_tablet_manager.cc:1398] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20240501 08:04:09.526343 17427 tablet_bootstrap.cc:492] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.527734 17312 raft_consensus.cc:3037] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.529099 17488 consensus_queue.cc:1035] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.531430 17427 tablet_bootstrap.cc:654] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.540218 17433 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.554358 17488 consensus_queue.cc:1035] T 19b58febdb8147b699a76df8ce1cb923 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.585664 17488 consensus_queue.cc:1035] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:04:09.607702 17278 consensus_peers.cc:485] T 298e75ee3b0345ea8766285d1ef4e09e P cdccadabd9154ea69e15405d75301fc1 -> Peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Couldn't send request to peer 5655c2e9168d4eeaa212db8874c2efa5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:09.633800 17464 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.668946 17426 tablet_bootstrap.cc:492] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.669410 17426 ts_tablet_manager.cc:1367] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.196s user 0.039s sys 0.036s
I20240501 08:04:09.671717 17426 raft_consensus.cc:348] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.672425 17426 raft_consensus.cc:374] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.672675 17426 raft_consensus.cc:729] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.673287 17426 consensus_queue.cc:260] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.677810 17428 tablet_bootstrap.cc:492] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.678339 17428 ts_tablet_manager.cc:1367] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.166s user 0.020s sys 0.052s
I20240501 08:04:09.680552 17428 raft_consensus.cc:348] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.681188 17428 raft_consensus.cc:374] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.681463 17428 raft_consensus.cc:729] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.682014 17428 consensus_queue.cc:260] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.675200 17426 ts_tablet_manager.cc:1398] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 08:04:09.683768 17426 tablet_bootstrap.cc:492] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.684491 17488 raft_consensus.cc:482] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.684922 17488 raft_consensus.cc:504] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.686677 17428 ts_tablet_manager.cc:1398] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240501 08:04:09.687126 17488 leader_election.cc:288] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.687495 17428 tablet_bootstrap.cc:492] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.688153 17488 raft_consensus.cc:482] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.688644 17488 raft_consensus.cc:504] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.690407 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.690871 17488 leader_election.cc:288] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.692138 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23486d7c47664dc89019c4d92d25e82d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.692309 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23486d7c47664dc89019c4d92d25e82d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.692735 17386 raft_consensus.cc:2454] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:09.692934 17237 raft_consensus.cc:2454] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:09.693306 17428 tablet_bootstrap.cc:654] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:09.693507 17278 leader_election.cc:341] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.694101 17277 leader_election.cc:302] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.694242 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.694921 17488 raft_consensus.cc:2781] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.695288 17488 raft_consensus.cc:482] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.695647 17488 raft_consensus.cc:3037] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.697434 17426 tablet_bootstrap.cc:654] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:09.701288 17277 leader_election.cc:341] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.701783 17277 leader_election.cc:302] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.706650 17495 raft_consensus.cc:2726] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.707440 17428 tablet_bootstrap.cc:492] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.707778 17428 ts_tablet_manager.cc:1367] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.020s user 0.015s sys 0.004s
I20240501 08:04:09.708321 17426 tablet_bootstrap.cc:492] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.708773 17426 ts_tablet_manager.cc:1367] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.008s
I20240501 08:04:09.709643 17428 raft_consensus.cc:348] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.710132 17428 raft_consensus.cc:374] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.710319 17428 raft_consensus.cc:729] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.710942 17428 consensus_queue.cc:260] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.711822 17426 raft_consensus.cc:348] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.712523 17426 raft_consensus.cc:374] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.712811 17426 raft_consensus.cc:729] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.713516 17426 consensus_queue.cc:260] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.715751 17426 ts_tablet_manager.cc:1398] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20240501 08:04:09.716598 17426 tablet_bootstrap.cc:492] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.716850 17427 tablet_bootstrap.cc:492] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.717265 17427 ts_tablet_manager.cc:1367] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.191s user 0.025s sys 0.062s
I20240501 08:04:09.717736 17428 ts_tablet_manager.cc:1398] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240501 08:04:09.718575 17428 tablet_bootstrap.cc:492] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
W20240501 08:04:09.719028 17351 consensus_peers.cc:485] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Couldn't send request to peer fa9d3bc74a68481e9de647223cb0110e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:09.719405 17427 raft_consensus.cc:348] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.720101 17427 raft_consensus.cc:374] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.720358 17427 raft_consensus.cc:729] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.720934 17427 consensus_queue.cc:260] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.722769 17488 raft_consensus.cc:504] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.723516 17428 tablet_bootstrap.cc:654] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.722538 17426 tablet_bootstrap.cc:654] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.724891 17488 leader_election.cc:288] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.725831 17427 ts_tablet_manager.cc:1398] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240501 08:04:09.726655 17427 tablet_bootstrap.cc:492] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.727599 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23486d7c47664dc89019c4d92d25e82d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:09.728215 17386 raft_consensus.cc:3037] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.731976 17427 tablet_bootstrap.cc:654] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.733582 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23486d7c47664dc89019c4d92d25e82d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:09.734146 17234 raft_consensus.cc:3037] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.734719 17428 tablet_bootstrap.cc:492] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.735309 17428 ts_tablet_manager.cc:1367] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.005s
I20240501 08:04:09.737568 17428 raft_consensus.cc:348] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.738195 17428 raft_consensus.cc:374] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.738446 17428 raft_consensus.cc:729] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.739259 17464 raft_consensus.cc:482] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.738994 17428 consensus_queue.cc:260] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.739713 17464 raft_consensus.cc:504] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.741218 17428 ts_tablet_manager.cc:1398] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 08:04:09.741605 17464 leader_election.cc:288] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.742091 17428 tablet_bootstrap.cc:492] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.743155 17427 tablet_bootstrap.cc:492] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.743201 17426 tablet_bootstrap.cc:492] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.743680 17427 ts_tablet_manager.cc:1367] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20240501 08:04:09.743546 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.743784 17426 ts_tablet_manager.cc:1367] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.005s
I20240501 08:04:09.744338 17312 raft_consensus.cc:2379] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:09.746287 17426 raft_consensus.cc:348] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.746475 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384fc32cbcd0455b9cce4f5e71f9a626" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.746912 17427 raft_consensus.cc:348] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.747705 17426 raft_consensus.cc:374] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.748019 17427 raft_consensus.cc:374] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.748088 17426 raft_consensus.cc:729] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.748306 17428 tablet_bootstrap.cc:654] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.748409 17427 raft_consensus.cc:729] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.748935 17426 consensus_queue.cc:260] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.749121 17427 consensus_queue.cc:260] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.751014 17427 ts_tablet_manager.cc:1398] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 08:04:09.751506 17426 ts_tablet_manager.cc:1398] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240501 08:04:09.751814 17427 tablet_bootstrap.cc:492] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.752390 17426 tablet_bootstrap.cc:492] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.752324 17203 leader_election.cc:302] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:09.744482 17504 raft_consensus.cc:482] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.753556 17504 raft_consensus.cc:504] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.755697 17504 leader_election.cc:288] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.757324 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.757920 17427 tablet_bootstrap.cc:654] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:09.758469 17351 leader_election.cc:341] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.758554 17466 raft_consensus.cc:3037] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.759483 17426 tablet_bootstrap.cc:654] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.762055 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.765302 17466 raft_consensus.cc:2726] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.766129 17234 raft_consensus.cc:2454] T 23486d7c47664dc89019c4d92d25e82d P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
W20240501 08:04:09.767949 17351 leader_election.cc:341] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.768268 17351 leader_election.cc:302] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.769467 17504 raft_consensus.cc:2726] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.769667 17426 tablet_bootstrap.cc:492] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.769783 17427 tablet_bootstrap.cc:492] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.770378 17427 ts_tablet_manager.cc:1367] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.006s
I20240501 08:04:09.770416 17426 ts_tablet_manager.cc:1367] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20240501 08:04:09.770464 17504 consensus_queue.cc:1035] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:09.771039 17428 tablet_bootstrap.cc:492] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.771445 17428 ts_tablet_manager.cc:1367] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.030s user 0.015s sys 0.001s
I20240501 08:04:09.772773 17427 raft_consensus.cc:348] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.772948 17426 raft_consensus.cc:348] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.773478 17427 raft_consensus.cc:374] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.773778 17427 raft_consensus.cc:729] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.773743 17426 raft_consensus.cc:374] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.773625 17428 raft_consensus.cc:348] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.774302 17426 raft_consensus.cc:729] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.774544 17428 raft_consensus.cc:374] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.774478 17427 consensus_queue.cc:260] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.774952 17428 raft_consensus.cc:729] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.775743 17277 leader_election.cc:302] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.775642 17428 consensus_queue.cc:260] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.776208 17504 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.776731 17427 ts_tablet_manager.cc:1398] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:09.776932 17519 raft_consensus.cc:2781] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.777374 17519 raft_consensus.cc:686] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:09.777578 17428 ts_tablet_manager.cc:1398] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:04:09.778414 17428 tablet_bootstrap.cc:492] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.778425 17519 consensus_queue.cc:237] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.777606 17427 tablet_bootstrap.cc:492] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.783988 17504 raft_consensus.cc:482] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.784433 17504 raft_consensus.cc:504] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.786819 17504 leader_election.cc:288] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.787556 17427 tablet_bootstrap.cc:654] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.782743 17386 raft_consensus.cc:2454] T 23486d7c47664dc89019c4d92d25e82d P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:09.775324 17426 consensus_queue.cc:260] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.795022 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.795632 17312 raft_consensus.cc:2454] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:09.790911 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.796697 17351 leader_election.cc:302] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:09.797413 17433 raft_consensus.cc:2781] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.797753 17433 raft_consensus.cc:482] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.798076 17433 raft_consensus.cc:3037] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.799612 17428 tablet_bootstrap.cc:654] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:09.798388 17351 leader_election.cc:341] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.797961 17426 ts_tablet_manager.cc:1398] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.027s user 0.004s sys 0.002s
I20240501 08:04:09.802455 17488 raft_consensus.cc:482] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.802888 17488 raft_consensus.cc:504] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.803678 17426 tablet_bootstrap.cc:492] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.804684 17488 leader_election.cc:288] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.807395 17504 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:09.813591 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cd3725c0f14e00a00542d3285b2d9d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.814391 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cd3725c0f14e00a00542d3285b2d9d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
W20240501 08:04:09.818053 17278 leader_election.cc:341] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.813177 17433 raft_consensus.cc:504] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.816942 17234 raft_consensus.cc:2454] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:09.820462 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:09.821633 17277 leader_election.cc:302] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:09.821738 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:09.822882 17311 raft_consensus.cc:3037] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.823032 17519 raft_consensus.cc:2781] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.823415 17519 raft_consensus.cc:482] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.823496 17433 leader_election.cc:288] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.823813 17519 raft_consensus.cc:3037] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.823498 17135 catalog_manager.cc:5484] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
W20240501 08:04:09.825284 17351 leader_election.cc:341] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.830273 17427 tablet_bootstrap.cc:492] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.830736 17427 ts_tablet_manager.cc:1367] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.053s user 0.005s sys 0.009s
I20240501 08:04:09.833428 17426 tablet_bootstrap.cc:654] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.833670 17464 raft_consensus.cc:482] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.834090 17464 raft_consensus.cc:504] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.836028 17464 leader_election.cc:288] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.833025 17427 raft_consensus.cc:348] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.836849 17427 raft_consensus.cc:374] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.837116 17427 raft_consensus.cc:729] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.837711 17427 consensus_queue.cc:260] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.838115 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.839144 17519 raft_consensus.cc:504] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.839648 17427 ts_tablet_manager.cc:1398] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20240501 08:04:09.839740 17466 raft_consensus.cc:482] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.840212 17466 raft_consensus.cc:504] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.841383 17519 leader_election.cc:288] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.842115 17466 leader_election.cc:288] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.845814 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d636a072366c4776aecc4b164749d410" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.846601 17504 raft_consensus.cc:482] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.847589 17504 raft_consensus.cc:504] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.848475 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cd3725c0f14e00a00542d3285b2d9d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:09.849560 17504 leader_election.cc:288] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:09.850795 17428 tablet_bootstrap.cc:492] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.851209 17428 ts_tablet_manager.cc:1367] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.007s
I20240501 08:04:09.850984 17309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5646a19da7ae4c4eafcc2532ee9326c3" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.851675 17309 raft_consensus.cc:2454] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
W20240501 08:04:09.852712 17278 leader_election.cc:341] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.852777 17309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe77234745b44d4983ec4508289a3a25" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.853662 17202 leader_election.cc:302] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.855484 17464 raft_consensus.cc:2781] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.856040 17464 raft_consensus.cc:482] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.856204 17428 raft_consensus.cc:348] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.856873 17428 raft_consensus.cc:374] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.857309 17428 raft_consensus.cc:729] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.857988 17428 consensus_queue.cc:260] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.858438 17311 raft_consensus.cc:2454] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:09.861486 17464 raft_consensus.cc:3037] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:09.865931 17351 leader_election.cc:341] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.867208 17351 leader_election.cc:302] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.868817 17503 raft_consensus.cc:2781] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.869203 17503 raft_consensus.cc:686] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:09.869912 17503 consensus_queue.cc:237] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.872066 17464 raft_consensus.cc:504] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.874034 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe77234745b44d4983ec4508289a3a25" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:09.875767 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cd3725c0f14e00a00542d3285b2d9d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:09.876359 17237 raft_consensus.cc:3037] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.879253 17428 ts_tablet_manager.cc:1398] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.027s user 0.003s sys 0.002s
I20240501 08:04:09.880165 17428 tablet_bootstrap.cc:492] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.883894 17237 raft_consensus.cc:2454] T f5cd3725c0f14e00a00542d3285b2d9d P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:09.884826 17428 tablet_bootstrap.cc:654] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.884977 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5646a19da7ae4c4eafcc2532ee9326c3" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.840592 17427 tablet_bootstrap.cc:492] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.889174 17133 catalog_manager.cc:5484] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:09.890841 17203 leader_election.cc:302] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:09.891577 17466 raft_consensus.cc:2726] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240501 08:04:09.893334 17203 leader_election.cc:341] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.898873 17427 tablet_bootstrap.cc:654] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.907478 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5646a19da7ae4c4eafcc2532ee9326c3" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:09.908150 17311 raft_consensus.cc:3037] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.924314 17464 leader_election.cc:288] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.927053 17311 raft_consensus.cc:2454] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:09.928375 17202 leader_election.cc:302] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.928871 17277 leader_election.cc:302] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
W20240501 08:04:09.929584 17351 leader_election.cc:341] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.929718 17519 raft_consensus.cc:2781] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.930227 17519 raft_consensus.cc:686] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:09.931644 17519 consensus_queue.cc:237] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.933072 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5646a19da7ae4c4eafcc2532ee9326c3" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:09.933708 17428 tablet_bootstrap.cc:492] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:09.933851 17426 tablet_bootstrap.cc:492] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.934113 17428 ts_tablet_manager.cc:1367] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.054s user 0.025s sys 0.020s
I20240501 08:04:09.934281 17426 ts_tablet_manager.cc:1367] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.131s user 0.053s sys 0.042s
W20240501 08:04:09.934442 17203 leader_election.cc:341] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.930091 17351 leader_election.cc:302] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:09.936561 17426 raft_consensus.cc:348] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.937127 17466 raft_consensus.cc:2781] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.937366 17433 raft_consensus.cc:2726] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:09.937687 17466 raft_consensus.cc:686] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:09.938222 17428 raft_consensus.cc:348] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.938906 17428 raft_consensus.cc:374] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.939170 17428 raft_consensus.cc:729] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.939085 17466 consensus_queue.cc:237] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.939827 17428 consensus_queue.cc:260] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.940327 17484 raft_consensus.cc:482] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.940758 17484 raft_consensus.cc:504] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.942849 17484 leader_election.cc:288] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.945099 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:09.945768 17386 raft_consensus.cc:2454] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:09.937215 17426 raft_consensus.cc:374] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.947818 17428 ts_tablet_manager.cc:1398] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20240501 08:04:09.947921 17203 leader_election.cc:302] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:09.947319 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:09.947845 17426 raft_consensus.cc:729] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
W20240501 08:04:09.949728 17202 leader_election.cc:341] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.949674 17426 consensus_queue.cc:260] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.948870 17484 raft_consensus.cc:2781] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:09.950704 17484 raft_consensus.cc:482] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:09.951025 17484 raft_consensus.cc:3037] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.951627 17426 ts_tablet_manager.cc:1398] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.017s user 0.002s sys 0.002s
I20240501 08:04:09.952443 17426 tablet_bootstrap.cc:492] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:09.948438 17427 tablet_bootstrap.cc:492] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:09.957269 17427 ts_tablet_manager.cc:1367] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.117s user 0.009s sys 0.009s
I20240501 08:04:09.957499 17426 tablet_bootstrap.cc:654] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.959098 17427 raft_consensus.cc:348] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.959575 17427 raft_consensus.cc:374] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.959779 17427 raft_consensus.cc:729] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.960276 17427 consensus_queue.cc:260] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.948820 17428 tablet_bootstrap.cc:492] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:09.961211 17484 raft_consensus.cc:504] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.963006 17484 leader_election.cc:288] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.964763 17427 ts_tablet_manager.cc:1398] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20240501 08:04:09.965629 17427 tablet_bootstrap.cc:492] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:09.966575 17428 tablet_bootstrap.cc:654] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.966740 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:09.967801 17133 catalog_manager.cc:5484] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
W20240501 08:04:09.969352 17202 leader_election.cc:341] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:09.967850 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:09.970427 17386 raft_consensus.cc:3037] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:09.970643 17427 tablet_bootstrap.cc:654] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:09.978188 17386 raft_consensus.cc:2454] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:09.979785 17203 leader_election.cc:302] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:09.981887 17484 raft_consensus.cc:2781] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:09.982324 17484 raft_consensus.cc:686] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:09.983027 17484 consensus_queue.cc:237] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.984359 17426 tablet_bootstrap.cc:492] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:09.984781 17426 ts_tablet_manager.cc:1367] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.011s
I20240501 08:04:09.986450 17464 raft_consensus.cc:482] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:09.986902 17464 raft_consensus.cc:504] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.987087 17426 raft_consensus.cc:348] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.987780 17426 raft_consensus.cc:374] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:09.988049 17426 raft_consensus.cc:729] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:09.988755 17464 leader_election.cc:288] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:09.988634 17426 consensus_queue.cc:260] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:09.990262 17426 ts_tablet_manager.cc:1398] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20240501 08:04:09.989609 17133 catalog_manager.cc:5484] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:09.991140 17426 tablet_bootstrap.cc:492] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.003790 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.004570 17311 raft_consensus.cc:2379] T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:09.996990 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81960706d26c481ba0b5441075d45c30" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.008996 17203 leader_election.cc:302] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:10.010207 17466 raft_consensus.cc:3037] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.016546 17466 raft_consensus.cc:2726] T 81960706d26c481ba0b5441075d45c30 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.019089 17427 tablet_bootstrap.cc:492] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.019587 17427 ts_tablet_manager.cc:1367] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.006s
I20240501 08:04:10.022562 17428 tablet_bootstrap.cc:492] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.022284 17427 raft_consensus.cc:348] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.022944 17427 raft_consensus.cc:374] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.022984 17428 ts_tablet_manager.cc:1367] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.074s user 0.036s sys 0.009s
I20240501 08:04:10.023303 17427 raft_consensus.cc:729] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.024004 17427 consensus_queue.cc:260] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.024672 17426 tablet_bootstrap.cc:654] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.025451 17428 raft_consensus.cc:348] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.026103 17428 raft_consensus.cc:374] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.026312 17427 ts_tablet_manager.cc:1398] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20240501 08:04:10.026369 17428 raft_consensus.cc:729] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.027228 17427 tablet_bootstrap.cc:492] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.027127 17428 consensus_queue.cc:260] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.033166 17427 tablet_bootstrap.cc:654] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.033447 17428 ts_tablet_manager.cc:1398] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20240501 08:04:10.037444 17428 tablet_bootstrap.cc:492] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.042398 17428 tablet_bootstrap.cc:654] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.047422 17427 tablet_bootstrap.cc:492] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.047843 17427 ts_tablet_manager.cc:1367] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.006s
I20240501 08:04:10.048280 17426 tablet_bootstrap.cc:492] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.048702 17426 ts_tablet_manager.cc:1367] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.015s
I20240501 08:04:10.050786 17433 raft_consensus.cc:482] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.051095 17426 raft_consensus.cc:348] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.051211 17433 raft_consensus.cc:504] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.051822 17426 raft_consensus.cc:374] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.052114 17426 raft_consensus.cc:729] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.052949 17428 tablet_bootstrap.cc:492] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.052696 17426 consensus_queue.cc:260] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.053370 17428 ts_tablet_manager.cc:1367] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20240501 08:04:10.054397 17427 raft_consensus.cc:348] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.055325 17427 raft_consensus.cc:374] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.055588 17427 raft_consensus.cc:729] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.056246 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77fc1fa8a74485a5698b48b0fa2472" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.056138 17427 consensus_queue.cc:260] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.056834 17311 raft_consensus.cc:2454] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:10.062006 17351 leader_election.cc:302] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:10.061798 17134 catalog_manager.cc:5484] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.062608 17503 raft_consensus.cc:2781] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.062958 17503 raft_consensus.cc:482] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.063277 17503 raft_consensus.cc:3037] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.056864 17428 raft_consensus.cc:348] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.054217 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77fc1fa8a74485a5698b48b0fa2472" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.054708 17433 leader_election.cc:288] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.064235 17428 raft_consensus.cc:374] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.064528 17428 raft_consensus.cc:729] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.065217 17428 consensus_queue.cc:260] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:10.065500 17351 leader_election.cc:341] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.067147 17428 ts_tablet_manager.cc:1398] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20240501 08:04:10.068722 17428 tablet_bootstrap.cc:492] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.069372 17503 raft_consensus.cc:504] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.058712 17426 ts_tablet_manager.cc:1398] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20240501 08:04:10.059147 17427 ts_tablet_manager.cc:1398] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20240501 08:04:10.072773 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77fc1fa8a74485a5698b48b0fa2472" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:10.073163 17427 tablet_bootstrap.cc:492] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.073336 17311 raft_consensus.cc:3037] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.074167 17428 tablet_bootstrap.cc:654] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.075219 17519 raft_consensus.cc:482] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.071835 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77fc1fa8a74485a5698b48b0fa2472" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:10.075664 17519 raft_consensus.cc:504] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.076947 17426 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
W20240501 08:04:10.077699 17351 leader_election.cc:341] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.079116 17519 leader_election.cc:288] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.081714 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe77234745b44d4983ec4508289a3a25" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.082866 17386 raft_consensus.cc:2454] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:10.082259 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe77234745b44d4983ec4508289a3a25" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.083815 17237 raft_consensus.cc:2454] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:10.084062 17278 leader_election.cc:302] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:10.084734 17427 tablet_bootstrap.cc:654] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.086655 17519 raft_consensus.cc:2781] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.087061 17519 raft_consensus.cc:482] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.089675 17426 tablet_bootstrap.cc:654] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.102835 17427 tablet_bootstrap.cc:492] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.089798 17311 raft_consensus.cc:2454] T 0d77fc1fa8a74485a5698b48b0fa2472 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:10.103374 17427 ts_tablet_manager.cc:1367] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.005s
I20240501 08:04:10.071828 17503 leader_election.cc:288] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.087386 17519 raft_consensus.cc:3037] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.104404 17428 tablet_bootstrap.cc:492] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.104864 17428 ts_tablet_manager.cc:1367] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.022s
I20240501 08:04:10.104781 17351 leader_election.cc:302] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.106065 17503 raft_consensus.cc:2781] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.107187 17433 raft_consensus.cc:482] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.107470 17428 raft_consensus.cc:348] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.108203 17428 raft_consensus.cc:374] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.107808 17433 raft_consensus.cc:504] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.108486 17428 raft_consensus.cc:729] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.109126 17428 consensus_queue.cc:260] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.110627 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e75ee3b0345ea8766285d1ef4e09e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.111203 17428 ts_tablet_manager.cc:1398] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:04:10.111724 17519 raft_consensus.cc:504] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.113413 17433 leader_election.cc:288] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.114665 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe77234745b44d4983ec4508289a3a25" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:10.115209 17237 raft_consensus.cc:3037] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.118453 17427 raft_consensus.cc:348] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.119606 17427 raft_consensus.cc:374] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.119864 17427 raft_consensus.cc:729] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.120518 17503 raft_consensus.cc:686] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:10.120442 17427 consensus_queue.cc:260] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.121105 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e75ee3b0345ea8766285d1ef4e09e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.121551 17519 leader_election.cc:288] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.122478 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe77234745b44d4983ec4508289a3a25" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:10.127602 17386 raft_consensus.cc:3037] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.127995 17351 leader_election.cc:302] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.129055 17433 raft_consensus.cc:2726] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.122748 17237 raft_consensus.cc:2454] T fe77234745b44d4983ec4508289a3a25 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:10.130929 17277 leader_election.cc:302] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:10.131704 17488 raft_consensus.cc:2781] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.125159 17503 consensus_queue.cc:237] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.132148 17488 raft_consensus.cc:686] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:10.133087 17488 consensus_queue.cc:237] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.134056 17386 raft_consensus.cc:2454] T fe77234745b44d4983ec4508289a3a25 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:10.134558 17427 ts_tablet_manager.cc:1398] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.031s user 0.005s sys 0.001s
I20240501 08:04:10.123793 17428 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.135509 17427 tablet_bootstrap.cc:492] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.140785 17426 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.141345 17428 tablet_bootstrap.cc:654] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.141888 17426 ts_tablet_manager.cc:1367] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.065s user 0.023s sys 0.024s
I20240501 08:04:10.143808 17427 tablet_bootstrap.cc:654] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.147527 17426 raft_consensus.cc:348] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.148249 17426 raft_consensus.cc:374] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.148550 17426 raft_consensus.cc:729] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.149258 17426 consensus_queue.cc:260] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.151211 17426 ts_tablet_manager.cc:1398] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20240501 08:04:10.152061 17426 tablet_bootstrap.cc:492] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.154237 17135 catalog_manager.cc:5484] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.155045 17428 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.157543 17426 tablet_bootstrap.cc:654] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.157869 17428 ts_tablet_manager.cc:1367] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.005s
I20240501 08:04:10.160355 17428 raft_consensus.cc:348] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.161187 17428 raft_consensus.cc:374] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.161486 17428 raft_consensus.cc:729] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.162119 17428 consensus_queue.cc:260] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.164265 17428 ts_tablet_manager.cc:1398] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240501 08:04:10.165194 17428 tablet_bootstrap.cc:492] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.168738 17133 catalog_manager.cc:5484] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:10.171079 17428 tablet_bootstrap.cc:654] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.185436 17427 tablet_bootstrap.cc:492] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.185881 17427 ts_tablet_manager.cc:1367] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.008s
I20240501 08:04:10.188148 17428 tablet_bootstrap.cc:492] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.188220 17427 raft_consensus.cc:348] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.188715 17428 ts_tablet_manager.cc:1367] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.004s
I20240501 08:04:10.188963 17427 raft_consensus.cc:374] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.189276 17427 raft_consensus.cc:729] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.189918 17427 consensus_queue.cc:260] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.191357 17427 ts_tablet_manager.cc:1398] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 08:04:10.191038 17428 raft_consensus.cc:348] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.191723 17428 raft_consensus.cc:374] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.191987 17428 raft_consensus.cc:729] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.193167 17427 tablet_bootstrap.cc:492] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.194389 17428 consensus_queue.cc:260] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.196535 17428 ts_tablet_manager.cc:1398] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20240501 08:04:10.198137 17426 tablet_bootstrap.cc:492] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.198549 17426 ts_tablet_manager.cc:1367] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.006s
I20240501 08:04:10.199173 17427 tablet_bootstrap.cc:654] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.201781 17426 raft_consensus.cc:348] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.202461 17426 raft_consensus.cc:374] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.202733 17426 raft_consensus.cc:729] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.202816 17428 tablet_bootstrap.cc:492] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.203414 17426 consensus_queue.cc:260] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.205400 17503 raft_consensus.cc:482] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.205861 17503 raft_consensus.cc:504] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.207661 17503 leader_election.cc:288] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.208465 17426 ts_tablet_manager.cc:1398] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20240501 08:04:10.208590 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.209308 17426 tablet_bootstrap.cc:492] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.208918 17464 raft_consensus.cc:482] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.209553 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.209857 17464 raft_consensus.cc:504] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:10.211522 17351 leader_election.cc:341] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.211588 17464 leader_election.cc:288] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.213155 17386 raft_consensus.cc:3037] T 298e75ee3b0345ea8766285d1ef4e09e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.214416 17428 tablet_bootstrap.cc:654] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.214628 17426 tablet_bootstrap.cc:654] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:10.221923 17351 leader_election.cc:341] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.222424 17351 leader_election.cc:302] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.223992 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.225255 17519 raft_consensus.cc:482] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.225705 17519 raft_consensus.cc:504] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.227205 17503 raft_consensus.cc:2726] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.227533 17519 leader_election.cc:288] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.229185 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4d63ea4cc54bb1962a1fe4c656ce95" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.229524 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c003e520523e4df5866a64c4612d1970" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.230705 17202 leader_election.cc:302] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:10.231755 17466 raft_consensus.cc:2726] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.231652 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c003e520523e4df5866a64c4612d1970" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.232868 17385 raft_consensus.cc:2454] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:10.233805 17278 leader_election.cc:302] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
W20240501 08:04:10.234299 17277 leader_election.cc:341] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.234548 17519 raft_consensus.cc:2781] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.234870 17519 raft_consensus.cc:482] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.235164 17519 raft_consensus.cc:3037] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.248636 17488 consensus_queue.cc:1035] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.241083 17519 raft_consensus.cc:504] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.251689 17519 leader_election.cc:288] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.252606 17503 raft_consensus.cc:482] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.253067 17503 raft_consensus.cc:504] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.253793 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c003e520523e4df5866a64c4612d1970" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:10.254402 17385 raft_consensus.cc:3037] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.255077 17503 leader_election.cc:288] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.258338 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5646a19da7ae4c4eafcc2532ee9326c3" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.259111 17311 raft_consensus.cc:2379] T 5646a19da7ae4c4eafcc2532ee9326c3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:10.259764 17466 raft_consensus.cc:482] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.260223 17466 raft_consensus.cc:504] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.262468 17466 leader_election.cc:288] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.264106 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38871d6cbe1846c89b09302d47294f87" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:10.265327 17202 leader_election.cc:341] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.267467 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38871d6cbe1846c89b09302d47294f87" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.268050 17386 raft_consensus.cc:2454] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:10.268653 17428 tablet_bootstrap.cc:492] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.269147 17428 ts_tablet_manager.cc:1367] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.066s user 0.020s sys 0.004s
I20240501 08:04:10.269294 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c003e520523e4df5866a64c4612d1970" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:10.269524 17427 tablet_bootstrap.cc:492] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.270067 17427 ts_tablet_manager.cc:1367] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.077s user 0.038s sys 0.010s
I20240501 08:04:10.272331 17427 raft_consensus.cc:348] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.272962 17427 raft_consensus.cc:374] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.273231 17427 raft_consensus.cc:729] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.273839 17427 consensus_queue.cc:260] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.275907 17427 ts_tablet_manager.cc:1398] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20240501 08:04:10.275971 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5646a19da7ae4c4eafcc2532ee9326c3" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.276784 17427 tablet_bootstrap.cc:492] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.279371 17351 leader_election.cc:302] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.280625 17433 raft_consensus.cc:3037] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.282538 17427 tablet_bootstrap.cc:654] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.282214 17428 raft_consensus.cc:348] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.282920 17428 raft_consensus.cc:374] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.283187 17428 raft_consensus.cc:729] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.283782 17428 consensus_queue.cc:260] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.285682 17428 ts_tablet_manager.cc:1398] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
W20240501 08:04:10.286495 17277 leader_election.cc:341] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.287631 17426 tablet_bootstrap.cc:492] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.288051 17426 ts_tablet_manager.cc:1367] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.079s user 0.020s sys 0.035s
I20240501 08:04:10.289698 17428 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.292490 17426 raft_consensus.cc:348] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.294881 17428 tablet_bootstrap.cc:654] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.295776 17488 consensus_queue.cc:1035] T 23486d7c47664dc89019c4d92d25e82d P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.299726 17427 tablet_bootstrap.cc:492] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.293154 17426 raft_consensus.cc:374] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.300176 17427 ts_tablet_manager.cc:1367] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.011s
I20240501 08:04:10.300204 17426 raft_consensus.cc:729] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.301133 17426 consensus_queue.cc:260] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.301848 17428 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.302248 17428 ts_tablet_manager.cc:1367] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20240501 08:04:10.303121 17426 ts_tablet_manager.cc:1398] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20240501 08:04:10.303333 17427 raft_consensus.cc:348] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.303943 17426 tablet_bootstrap.cc:492] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.303994 17427 raft_consensus.cc:374] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.304358 17427 raft_consensus.cc:729] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.304443 17428 raft_consensus.cc:348] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.305136 17428 raft_consensus.cc:374] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.304953 17427 consensus_queue.cc:260] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.305433 17428 raft_consensus.cc:729] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.306499 17432 raft_consensus.cc:482] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.306401 17428 consensus_queue.cc:260] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.307022 17427 ts_tablet_manager.cc:1398] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20240501 08:04:10.306942 17432 raft_consensus.cc:504] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.307832 17427 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.308519 17428 ts_tablet_manager.cc:1398] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:10.308995 17432 leader_election.cc:288] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.309371 17428 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.309728 17432 raft_consensus.cc:482] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.310160 17432 raft_consensus.cc:504] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.310938 17433 raft_consensus.cc:2726] T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.311921 17432 leader_election.cc:288] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.317821 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.315055 17203 leader_election.cc:302] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:10.319418 17384 raft_consensus.cc:2379] T 37a8e259653a4dc28c1c6875ec8b520a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:10.319730 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7567a8d257c4c908317642193d10087" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
W20240501 08:04:10.321635 17278 leader_election.cc:341] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.322366 17466 raft_consensus.cc:2781] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.322723 17466 raft_consensus.cc:482] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.323041 17466 raft_consensus.cc:3037] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.324729 17385 raft_consensus.cc:2454] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:10.325842 17278 leader_election.cc:302] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.327306 17432 raft_consensus.cc:2781] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.327731 17432 raft_consensus.cc:686] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:10.328488 17428 tablet_bootstrap.cc:654] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:10.329386 17203 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.131:43915, user_credentials={real_user=slave}} blocked reactor thread for 50435.1us
I20240501 08:04:10.328483 17432 consensus_queue.cc:237] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.331629 17466 raft_consensus.cc:504] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.332208 17426 tablet_bootstrap.cc:654] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.332859 17235 raft_consensus.cc:3037] T 9f4d63ea4cc54bb1962a1fe4c656ce95 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.340770 17464 raft_consensus.cc:482] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.344197 17427 tablet_bootstrap.cc:654] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.346349 17466 leader_election.cc:288] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.341310 17464 raft_consensus.cc:504] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.342509 17484 consensus_queue.cc:1035] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20240501 08:04:10.350937 17464 leader_election.cc:288] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.355851 17426 tablet_bootstrap.cc:492] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.356276 17426 ts_tablet_manager.cc:1367] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.052s user 0.006s sys 0.021s
I20240501 08:04:10.359045 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38871d6cbe1846c89b09302d47294f87" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:10.359697 17385 raft_consensus.cc:3037] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.361617 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c003e520523e4df5866a64c4612d1970" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.362424 17386 raft_consensus.cc:2379] T c003e520523e4df5866a64c4612d1970 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:10.363015 17234 raft_consensus.cc:3037] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.371038 17386 raft_consensus.cc:3037] T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.377965 17133 catalog_manager.cc:5484] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.378944 17426 raft_consensus.cc:348] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.379887 17426 raft_consensus.cc:374] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.380108 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38871d6cbe1846c89b09302d47294f87" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:10.380726 17426 raft_consensus.cc:729] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.381491 17426 consensus_queue.cc:260] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.382871 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c003e520523e4df5866a64c4612d1970" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:10.382885 17202 leader_election.cc:341] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.384734 17426 ts_tablet_manager.cc:1398] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.028s user 0.008s sys 0.000s
I20240501 08:04:10.391364 17236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7567a8d257c4c908317642193d10087" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.392010 17236 raft_consensus.cc:2454] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:10.391299 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a8e259653a4dc28c1c6875ec8b520a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.391230 17433 consensus_queue.cc:1035] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240501 08:04:10.395761 17277 leader_election.cc:302] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:10.397679 17519 raft_consensus.cc:2781] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.396906 17277 leader_election.cc:302] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.398711 17519 raft_consensus.cc:482] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.398133 17427 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.399111 17519 raft_consensus.cc:3037] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.399557 17427 ts_tablet_manager.cc:1367] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.092s user 0.021s sys 0.019s
I20240501 08:04:10.398850 17426 tablet_bootstrap.cc:492] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.399827 17488 raft_consensus.cc:3037] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.398427 17432 raft_consensus.cc:482] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.406072 17202 leader_election.cc:302] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:10.406512 17432 raft_consensus.cc:504] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.406973 17433 raft_consensus.cc:482] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.407387 17433 raft_consensus.cc:504] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.408315 17432 leader_election.cc:288] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.409703 17464 consensus_queue.cc:1035] T 5646a19da7ae4c4eafcc2532ee9326c3 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.410221 17385 raft_consensus.cc:2454] T 38871d6cbe1846c89b09302d47294f87 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:10.410977 17464 raft_consensus.cc:3037] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.416301 17203 leader_election.cc:302] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:10.416399 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ca37e5ca8844c68b9522e5d05bfc3e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.417097 17235 raft_consensus.cc:2454] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:10.418601 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ca37e5ca8844c68b9522e5d05bfc3e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.419996 17484 raft_consensus.cc:2781] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.420516 17484 raft_consensus.cc:686] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:10.420768 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87f059d40b441d5ad84b2e410829e61" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.422295 17385 raft_consensus.cc:2454] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:10.421916 17484 consensus_queue.cc:237] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.419239 17488 raft_consensus.cc:2726] T 37a8e259653a4dc28c1c6875ec8b520a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.412751 17433 leader_election.cc:288] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.420069 17312 raft_consensus.cc:2454] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:10.426142 17351 leader_election.cc:302] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:10.426278 17278 leader_election.cc:302] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:10.426946 17433 raft_consensus.cc:2781] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.426648 17519 raft_consensus.cc:504] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.427307 17433 raft_consensus.cc:482] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.427651 17433 raft_consensus.cc:3037] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.427872 17488 raft_consensus.cc:2781] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.428248 17488 raft_consensus.cc:482] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.428558 17488 raft_consensus.cc:3037] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.429641 17519 leader_election.cc:288] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.420305 17432 consensus_queue.cc:1035] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.430338 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7567a8d257c4c908317642193d10087" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
W20240501 08:04:10.438076 17278 leader_election.cc:341] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.450569 17605 raft_consensus.cc:482] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.451130 17605 raft_consensus.cc:504] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.453167 17605 leader_election.cc:288] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.467383 17519 raft_consensus.cc:482] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.443135 17464 raft_consensus.cc:2726] T c003e520523e4df5866a64c4612d1970 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240501 08:04:10.476020 17601 log.cc:924] Time spent T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1: Append to log took a long time: real 0.103s user 0.001s sys 0.000s
I20240501 08:04:10.468761 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87f059d40b441d5ad84b2e410829e61" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.489706 17427 raft_consensus.cc:348] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.490398 17427 raft_consensus.cc:374] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.490653 17427 raft_consensus.cc:729] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.491302 17427 consensus_queue.cc:260] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.468061 17519 raft_consensus.cc:504] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.501518 17428 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.502681 17428 ts_tablet_manager.cc:1367] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.193s user 0.021s sys 0.003s
I20240501 08:04:10.503351 17427 ts_tablet_manager.cc:1398] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.101s user 0.006s sys 0.013s
I20240501 08:04:10.504289 17427 tablet_bootstrap.cc:492] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.510427 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb40f0013fc419fb843b002645ed74d" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.511976 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7567a8d257c4c908317642193d10087" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:10.512411 17384 raft_consensus.cc:2454] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:10.512578 17233 raft_consensus.cc:3037] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.513028 17519 leader_election.cc:288] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:10.506610 17277 leader_election.cc:341] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.519503 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b993df33624a4500a73b663ca80d75f7" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.523810 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb40f0013fc419fb843b002645ed74d" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.507952 17433 raft_consensus.cc:504] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:10.528349 17278 leader_election.cc:341] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.531002 17433 leader_election.cc:288] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.536703 17504 consensus_queue.cc:1035] T 81960706d26c481ba0b5441075d45c30 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.538965 17426 tablet_bootstrap.cc:654] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.542088 17488 raft_consensus.cc:504] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.526216 17203 leader_election.cc:302] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
W20240501 08:04:10.545320 17202 leader_election.cc:341] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.545378 17605 raft_consensus.cc:2781] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.545800 17605 raft_consensus.cc:482] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.546126 17605 raft_consensus.cc:3037] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.549082 17488 leader_election.cc:288] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.549973 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ca37e5ca8844c68b9522e5d05bfc3e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:10.550801 17312 raft_consensus.cc:3037] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.553493 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87f059d40b441d5ad84b2e410829e61" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:10.554068 17384 raft_consensus.cc:3037] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.558933 17503 raft_consensus.cc:482] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.559394 17503 raft_consensus.cc:504] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.560416 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57ca37e5ca8844c68b9522e5d05bfc3e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:10.560993 17237 raft_consensus.cc:3037] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.564167 17229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b47c5f05da46bc9d9b5b4b202c1210" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.564309 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b993df33624a4500a73b663ca80d75f7" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.565048 17503 leader_election.cc:288] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:10.567672 17484 consensus_queue.cc:1035] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:04:10.570353 17277 leader_election.cc:341] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.570803 17277 leader_election.cc:302] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.572691 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87f059d40b441d5ad84b2e410829e61" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
W20240501 08:04:10.565802 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:10.563153 17312 raft_consensus.cc:2454] T 57ca37e5ca8844c68b9522e5d05bfc3e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
W20240501 08:04:10.579250 17351 leader_election.cc:341] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.580581 17432 consensus_queue.cc:1035] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.573316 17232 raft_consensus.cc:3037] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.590533 17605 raft_consensus.cc:504] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.592823 17432 consensus_queue.cc:1035] T f5cd3725c0f14e00a00542d3285b2d9d P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.594573 17605 leader_election.cc:288] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.591581 17427 tablet_bootstrap.cc:654] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.592355 17237 raft_consensus.cc:2454] T 57ca37e5ca8844c68b9522e5d05bfc3e P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:10.600566 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b47c5f05da46bc9d9b5b4b202c1210" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:10.596237 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb40f0013fc419fb843b002645ed74d" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:10.601261 17383 raft_consensus.cc:3037] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.601737 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb40f0013fc419fb843b002645ed74d" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:10.602715 17351 leader_election.cc:302] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:10.603986 17455 raft_consensus.cc:2781] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.578272 17519 raft_consensus.cc:2726] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.604439 17455 raft_consensus.cc:686] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:10.611649 17133 catalog_manager.cc:5484] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.618041 17455 consensus_queue.cc:237] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:10.608687 17351 leader_election.cc:341] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:04:10.613384 17202 leader_election.cc:341] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.633903 17428 raft_consensus.cc:348] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.634702 17428 raft_consensus.cc:374] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.634985 17428 raft_consensus.cc:729] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.635579 17428 consensus_queue.cc:260] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.607719 17503 consensus_queue.cc:1035] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20240501 08:04:10.636905 17351 leader_election.cc:302] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.649444 17428 ts_tablet_manager.cc:1398] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.146s user 0.007s sys 0.000s
I20240501 08:04:10.650488 17428 tablet_bootstrap.cc:492] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:10.649922 17504 raft_consensus.cc:2726] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:10.653757 17384 raft_consensus.cc:2454] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:10.654601 17383 raft_consensus.cc:2454] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
W20240501 08:04:10.659798 17608 log.cc:924] Time spent T f5cd3725c0f14e00a00542d3285b2d9d P 5655c2e9168d4eeaa212db8874c2efa5: Append to log took a long time: real 0.103s user 0.001s sys 0.000s
I20240501 08:04:10.662992 17426 tablet_bootstrap.cc:492] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.663420 17426 ts_tablet_manager.cc:1367] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.265s user 0.024s sys 0.033s
W20240501 08:04:10.664016 17607 log.cc:924] Time spent T 5646a19da7ae4c4eafcc2532ee9326c3 P 5655c2e9168d4eeaa212db8874c2efa5: Append to log took a long time: real 0.094s user 0.002s sys 0.000s
I20240501 08:04:10.670769 17426 raft_consensus.cc:348] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.671273 17426 raft_consensus.cc:374] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.671483 17426 raft_consensus.cc:729] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.671985 17426 consensus_queue.cc:260] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.673684 17426 ts_tablet_manager.cc:1398] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.004s sys 0.001s
I20240501 08:04:10.674508 17426 tablet_bootstrap.cc:492] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.679539 17428 tablet_bootstrap.cc:654] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:10.664299 17351 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.168.130:33757, user_credentials={real_user=slave}} blocked reactor thread for 55958.4us
I20240501 08:04:10.682935 17203 leader_election.cc:302] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:10.675515 17278 leader_election.cc:302] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:10.676668 17233 raft_consensus.cc:2454] T c7567a8d257c4c908317642193d10087 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:10.687091 17484 raft_consensus.cc:2781] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.693863 17519 raft_consensus.cc:2781] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.694056 17484 raft_consensus.cc:686] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:10.697695 17519 raft_consensus.cc:686] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:10.695012 17484 consensus_queue.cc:237] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.702041 17519 consensus_queue.cc:237] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.698891 17464 consensus_queue.cc:1035] T 37a8e259653a4dc28c1c6875ec8b520a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20240501 08:04:10.731233 17277 leader_election.cc:302] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:10.732578 17495 raft_consensus.cc:2781] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:10.778489 17495 raft_consensus.cc:686] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:10.807714 17427 tablet_bootstrap.cc:492] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.808368 17427 ts_tablet_manager.cc:1367] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.304s user 0.030s sys 0.026s
I20240501 08:04:10.780920 17232 raft_consensus.cc:2454] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
W20240501 08:04:10.783121 17278 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.131:43915, user_credentials={real_user=slave}} blocked reactor thread for 46136.9us
W20240501 08:04:10.808760 17611 log.cc:924] Time spent T 81960706d26c481ba0b5441075d45c30 P cdccadabd9154ea69e15405d75301fc1: Append to log took a long time: real 0.069s user 0.000s sys 0.001s
I20240501 08:04:10.783666 17488 consensus_queue.cc:1035] T fe77234745b44d4983ec4508289a3a25 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20240501 08:04:10.821871 17426 tablet_bootstrap.cc:654] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.833022 17428 tablet_bootstrap.cc:492] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:10.833508 17428 ts_tablet_manager.cc:1367] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.183s user 0.013s sys 0.002s
I20240501 08:04:10.828644 17427 raft_consensus.cc:348] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.834182 17427 raft_consensus.cc:374] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.835251 17427 raft_consensus.cc:729] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.835750 17427 consensus_queue.cc:260] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.846812 17428 raft_consensus.cc:348] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.847527 17428 raft_consensus.cc:374] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.847815 17428 raft_consensus.cc:729] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.848512 17428 consensus_queue.cc:260] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.850268 17428 ts_tablet_manager.cc:1398] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20240501 08:04:10.851903 17428 tablet_bootstrap.cc:492] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
W20240501 08:04:10.851027 17351 consensus_peers.cc:485] T 0d77fc1fa8a74485a5698b48b0fa2472 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Couldn't send request to peer fa9d3bc74a68481e9de647223cb0110e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:10.854468 17135 catalog_manager.cc:5484] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:10.858034 17623 raft_consensus.cc:482] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.858548 17623 raft_consensus.cc:504] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.861735 17427 ts_tablet_manager.cc:1398] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.053s user 0.004s sys 0.002s
I20240501 08:04:10.862807 17427 tablet_bootstrap.cc:492] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.822512 17495 consensus_queue.cc:237] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.879961 17623 leader_election.cc:288] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.897140 17426 tablet_bootstrap.cc:492] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:10.897609 17426 ts_tablet_manager.cc:1367] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.223s user 0.012s sys 0.003s
I20240501 08:04:10.897482 17625 consensus_queue.cc:1035] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20240501 08:04:10.900682 17426 raft_consensus.cc:348] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.901335 17426 raft_consensus.cc:374] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.901576 17426 raft_consensus.cc:729] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.902061 17426 consensus_queue.cc:260] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.906286 17427 tablet_bootstrap.cc:654] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.905805 17132 catalog_manager.cc:5484] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.907465 17135 catalog_manager.cc:5484] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.908536 17135 catalog_manager.cc:5484] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:10.911954 17426 ts_tablet_manager.cc:1398] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20240501 08:04:10.912847 17426 tablet_bootstrap.cc:492] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:10.914294 17428 tablet_bootstrap.cc:654] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.915902 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:10.916945 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.918004 17381 raft_consensus.cc:2454] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:10.923223 17625 consensus_queue.cc:1035] T c003e520523e4df5866a64c4612d1970 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:10.926625 17619 consensus_queue.cc:1035] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20240501 08:04:10.928241 17277 leader_election.cc:341] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.933077 17427 tablet_bootstrap.cc:492] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.933547 17427 ts_tablet_manager.cc:1367] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.071s user 0.007s sys 0.018s
I20240501 08:04:10.935818 17427 raft_consensus.cc:348] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.936487 17427 raft_consensus.cc:374] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.936759 17427 raft_consensus.cc:729] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.937876 17427 consensus_queue.cc:260] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.940495 17426 tablet_bootstrap.cc:654] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.941175 17427 ts_tablet_manager.cc:1398] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20240501 08:04:10.946748 17427 tablet_bootstrap.cc:492] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:10.956521 17427 tablet_bootstrap.cc:654] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:10.965692 17466 raft_consensus.cc:482] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:10.966264 17466 raft_consensus.cc:504] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.968385 17466 leader_election.cc:288] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.970882 17427 tablet_bootstrap.cc:492] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:10.971045 17382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b993df33624a4500a73b663ca80d75f7" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:10.971354 17427 ts_tablet_manager.cc:1367] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.004s
W20240501 08:04:10.975224 17203 leader_election.cc:341] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.973621 17427 raft_consensus.cc:348] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.978842 17427 raft_consensus.cc:374] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:10.979166 17427 raft_consensus.cc:729] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:10.980968 17427 consensus_queue.cc:260] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.982949 17278 leader_election.cc:302] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:10.983794 17495 raft_consensus.cc:2781] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.984141 17495 raft_consensus.cc:482] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.984447 17495 raft_consensus.cc:3037] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.990464 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b993df33624a4500a73b663ca80d75f7" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:10.990926 17202 consensus_peers.cc:485] T 38871d6cbe1846c89b09302d47294f87 P fa9d3bc74a68481e9de647223cb0110e -> Peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Couldn't send request to peer cdccadabd9154ea69e15405d75301fc1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:10.991117 17311 raft_consensus.cc:2454] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:10.992347 17202 leader_election.cc:302] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:10.992046 17495 raft_consensus.cc:504] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:10.994627 17484 raft_consensus.cc:2781] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.994988 17484 raft_consensus.cc:482] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:10.995334 17484 raft_consensus.cc:3037] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:10.995344 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
W20240501 08:04:10.996778 17277 leader_election.cc:341] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:10.997695 17495 leader_election.cc:288] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:10.999696 17599 raft_consensus.cc:482] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.000201 17599 raft_consensus.cc:504] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.002204 17484 raft_consensus.cc:504] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.003078 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c1e10c92f744c3a22dec7eecae1040" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.003674 17381 raft_consensus.cc:2454] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.004686 17203 leader_election.cc:302] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.005466 17619 raft_consensus.cc:2781] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:10.982939 17427 ts_tablet_manager.cc:1398] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20240501 08:04:11.005812 17619 raft_consensus.cc:482] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.006728 17619 raft_consensus.cc:3037] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.007737 17427 tablet_bootstrap.cc:492] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.009173 17484 leader_election.cc:288] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.011755 17488 raft_consensus.cc:482] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.012189 17488 raft_consensus.cc:504] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.014122 17488 leader_election.cc:288] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.014961 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b993df33624a4500a73b663ca80d75f7" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.015187 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c1e10c92f744c3a22dec7eecae1040" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.015801 17311 raft_consensus.cc:2454] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
W20240501 08:04:11.016706 17203 leader_election.cc:341] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.017100 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b993df33624a4500a73b663ca80d75f7" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.017653 17311 raft_consensus.cc:3037] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.020143 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.020704 17381 raft_consensus.cc:3037] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.021549 17234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb40f0013fc419fb843b002645ed74d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.006523 17599 leader_election.cc:288] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.041693 17311 raft_consensus.cc:2454] T b993df33624a4500a73b663ca80d75f7 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.045702 17619 raft_consensus.cc:504] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.047478 17428 tablet_bootstrap.cc:492] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:11.047930 17428 ts_tablet_manager.cc:1367] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.196s user 0.016s sys 0.024s
I20240501 08:04:11.047981 17619 leader_election.cc:288] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.048959 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c1e10c92f744c3a22dec7eecae1040" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.049582 17385 raft_consensus.cc:3037] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.050467 17428 raft_consensus.cc:348] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.050994 17428 raft_consensus.cc:374] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.051215 17428 raft_consensus.cc:729] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.051765 17428 consensus_queue.cc:260] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.053436 17428 ts_tablet_manager.cc:1398] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20240501 08:04:11.054527 17428 tablet_bootstrap.cc:492] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:11.054685 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb40f0013fc419fb843b002645ed74d" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.055430 17386 raft_consensus.cc:2379] T 4eb40f0013fc419fb843b002645ed74d P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:11.056011 17427 tablet_bootstrap.cc:654] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.048249 17381 raft_consensus.cc:2454] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:11.058595 17426 tablet_bootstrap.cc:492] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.059005 17426 ts_tablet_manager.cc:1367] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.146s user 0.029s sys 0.033s
I20240501 08:04:11.059651 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c1e10c92f744c3a22dec7eecae1040" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.053372 17202 leader_election.cc:302] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:11.060217 17308 raft_consensus.cc:3037] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.062239 17385 raft_consensus.cc:2454] T 85c1e10c92f744c3a22dec7eecae1040 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.063271 17203 leader_election.cc:302] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.064095 17484 raft_consensus.cc:2781] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.061260 17426 raft_consensus.cc:348] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.064538 17426 raft_consensus.cc:374] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.064652 17599 raft_consensus.cc:2781] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.064822 17426 raft_consensus.cc:729] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.065153 17599 raft_consensus.cc:686] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:11.065507 17426 consensus_queue.cc:260] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.065944 17599 consensus_queue.cc:237] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.066776 17308 raft_consensus.cc:2454] T 85c1e10c92f744c3a22dec7eecae1040 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.067442 17426 ts_tablet_manager.cc:1398] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240501 08:04:11.068464 17426 tablet_bootstrap.cc:492] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.070384 17484 raft_consensus.cc:686] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:11.073995 17428 tablet_bootstrap.cc:654] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.073681 17484 consensus_queue.cc:237] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.076253 17278 leader_election.cc:302] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.080066 17426 tablet_bootstrap.cc:654] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.087821 17432 raft_consensus.cc:3037] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.094741 17432 raft_consensus.cc:2726] T 4eb40f0013fc419fb843b002645ed74d P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.096114 17278 leader_election.cc:302] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.099159 17488 raft_consensus.cc:2781] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.099673 17488 raft_consensus.cc:686] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:11.100483 17488 consensus_queue.cc:237] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.101634 17426 tablet_bootstrap.cc:492] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.102196 17426 ts_tablet_manager.cc:1367] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.010s
I20240501 08:04:11.104717 17426 raft_consensus.cc:348] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.105562 17426 raft_consensus.cc:374] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.105875 17426 raft_consensus.cc:729] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.106542 17426 consensus_queue.cc:260] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.108642 17426 ts_tablet_manager.cc:1398] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20240501 08:04:11.109850 17426 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.115123 17426 tablet_bootstrap.cc:654] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.115165 17132 catalog_manager.cc:5484] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:11.117440 17132 catalog_manager.cc:5484] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:11.127740 17426 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.128155 17426 ts_tablet_manager.cc:1367] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s
I20240501 08:04:11.130393 17426 raft_consensus.cc:348] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.131003 17426 raft_consensus.cc:374] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.131229 17428 tablet_bootstrap.cc:492] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:11.131660 17428 ts_tablet_manager.cc:1367] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.077s user 0.025s sys 0.019s
I20240501 08:04:11.131604 17504 raft_consensus.cc:482] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.132134 17504 raft_consensus.cc:504] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.133586 17427 tablet_bootstrap.cc:492] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.133971 17504 leader_election.cc:288] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.134003 17427 ts_tablet_manager.cc:1367] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.126s user 0.024s sys 0.023s
I20240501 08:04:11.134479 17428 raft_consensus.cc:348] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.135149 17428 raft_consensus.cc:374] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.135404 17428 raft_consensus.cc:729] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.131242 17426 raft_consensus.cc:729] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.135649 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726057013c1e48f985e1fd55e502c0e0" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.136570 17426 consensus_queue.cc:260] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.138068 17426 ts_tablet_manager.cc:1398] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20240501 08:04:11.138263 17351 leader_election.cc:341] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.138389 17427 raft_consensus.cc:348] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.139047 17427 raft_consensus.cc:374] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.139315 17427 raft_consensus.cc:729] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.139889 17427 consensus_queue.cc:260] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.142091 17427 ts_tablet_manager.cc:1398] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240501 08:04:11.144253 17427 tablet_bootstrap.cc:492] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.146709 17132 catalog_manager.cc:5484] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:11.150159 17427 tablet_bootstrap.cc:654] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.150199 17426 tablet_bootstrap.cc:492] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.138505 17428 consensus_queue.cc:260] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.151010 17504 consensus_queue.cc:1035] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.152503 17428 ts_tablet_manager.cc:1398] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20240501 08:04:11.153360 17428 tablet_bootstrap.cc:492] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:11.158207 17428 tablet_bootstrap.cc:654] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.158491 17426 tablet_bootstrap.cc:654] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.169052 17428 tablet_bootstrap.cc:492] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:11.169514 17428 ts_tablet_manager.cc:1367] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.006s
I20240501 08:04:11.171949 17428 raft_consensus.cc:348] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.172609 17428 raft_consensus.cc:374] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.172866 17428 raft_consensus.cc:729] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.173722 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726057013c1e48f985e1fd55e502c0e0" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.173645 17428 consensus_queue.cc:260] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:11.176761 17351 leader_election.cc:341] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.177156 17351 leader_election.cc:302] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.178153 17455 raft_consensus.cc:2726] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.179589 17428 ts_tablet_manager.cc:1398] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240501 08:04:11.182111 17428 tablet_bootstrap.cc:492] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:11.189421 17427 tablet_bootstrap.cc:492] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.189867 17427 ts_tablet_manager.cc:1367] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.046s user 0.027s sys 0.007s
I20240501 08:04:11.191833 17428 tablet_bootstrap.cc:654] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.192873 17599 consensus_queue.cc:1035] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.196334 17427 raft_consensus.cc:348] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.197000 17427 raft_consensus.cc:374] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.197288 17427 raft_consensus.cc:729] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.197849 17427 consensus_queue.cc:260] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.199837 17427 ts_tablet_manager.cc:1398] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240501 08:04:11.200636 17427 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.205906 17427 tablet_bootstrap.cc:654] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.208555 17426 tablet_bootstrap.cc:492] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.209064 17426 ts_tablet_manager.cc:1367] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.009s
I20240501 08:04:11.211587 17426 raft_consensus.cc:348] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.212280 17426 raft_consensus.cc:374] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.212528 17426 raft_consensus.cc:729] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.213981 17426 consensus_queue.cc:260] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.216008 17426 ts_tablet_manager.cc:1398] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20240501 08:04:11.216773 17426 tablet_bootstrap.cc:492] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.221066 17427 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.221504 17427 ts_tablet_manager.cc:1367] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.006s
I20240501 08:04:11.223208 17427 raft_consensus.cc:348] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.223738 17427 raft_consensus.cc:374] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.223939 17427 raft_consensus.cc:729] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.224460 17427 consensus_queue.cc:260] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.225077 17484 raft_consensus.cc:482] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.225514 17484 raft_consensus.cc:504] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.227629 17484 leader_election.cc:288] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.228834 17426 tablet_bootstrap.cc:654] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.229254 17427 ts_tablet_manager.cc:1398] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20240501 08:04:11.230396 17427 tablet_bootstrap.cc:492] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.230540 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b47c5f05da46bc9d9b5b4b202c1210" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.231164 17385 raft_consensus.cc:2454] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.232210 17484 consensus_queue.cc:1035] T 4eb40f0013fc419fb843b002645ed74d P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.235225 17427 tablet_bootstrap.cc:654] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.237107 17426 tablet_bootstrap.cc:492] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.237591 17426 ts_tablet_manager.cc:1367] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.000s
I20240501 08:04:11.237813 17203 leader_election.cc:302] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.239758 17426 raft_consensus.cc:348] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.240895 17484 raft_consensus.cc:2781] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.241312 17484 raft_consensus.cc:482] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.241686 17484 raft_consensus.cc:3037] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.240417 17426 raft_consensus.cc:374] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.247125 17426 raft_consensus.cc:729] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.247741 17426 consensus_queue.cc:260] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.248805 17427 tablet_bootstrap.cc:492] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.249184 17427 ts_tablet_manager.cc:1367] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.008s
I20240501 08:04:11.249305 17599 raft_consensus.cc:482] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.249733 17599 raft_consensus.cc:504] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.251626 17599 leader_election.cc:288] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.251538 17427 raft_consensus.cc:348] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.252244 17427 raft_consensus.cc:374] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.252516 17427 raft_consensus.cc:729] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.249794 17426 ts_tablet_manager.cc:1398] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20240501 08:04:11.253170 17233 raft_consensus.cc:3037] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.253150 17427 consensus_queue.cc:260] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.254513 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77fc1fa8a74485a5698b48b0fa2472" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.255420 17427 ts_tablet_manager.cc:1398] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20240501 08:04:11.256204 17427 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.261456 17427 tablet_bootstrap.cc:654] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.267308 17432 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:11.267712 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b47c5f05da46bc9d9b5b4b202c1210" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.270140 17308 raft_consensus.cc:2454] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.270641 17426 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.271802 17427 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.272176 17427 ts_tablet_manager.cc:1367] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.007s
I20240501 08:04:11.274446 17427 raft_consensus.cc:348] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.275137 17427 raft_consensus.cc:374] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.274853 17503 consensus_queue.cc:1035] T 57ca37e5ca8844c68b9522e5d05bfc3e P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.275444 17427 raft_consensus.cc:729] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.276062 17427 consensus_queue.cc:260] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.277364 17426 tablet_bootstrap.cc:654] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.294798 17427 ts_tablet_manager.cc:1398] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.022s user 0.005s sys 0.002s
I20240501 08:04:11.295866 17427 tablet_bootstrap.cc:492] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.305568 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d77fc1fa8a74485a5698b48b0fa2472" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.306438 17426 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.306947 17426 ts_tablet_manager.cc:1367] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.036s user 0.014s sys 0.003s
I20240501 08:04:11.309270 17488 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.311533 17202 leader_election.cc:302] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:11.314594 17426 raft_consensus.cc:348] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.315493 17427 tablet_bootstrap.cc:654] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.315452 17426 raft_consensus.cc:374] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.315917 17426 raft_consensus.cc:729] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.316731 17426 consensus_queue.cc:260] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.319051 17426 ts_tablet_manager.cc:1398] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20240501 08:04:11.320833 17426 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.324677 17484 raft_consensus.cc:504] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.326414 17426 tablet_bootstrap.cc:654] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.327798 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b47c5f05da46bc9d9b5b4b202c1210" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.328452 17381 raft_consensus.cc:3037] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.336536 17663 raft_consensus.cc:2726] T 0d77fc1fa8a74485a5698b48b0fa2472 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.337956 17663 raft_consensus.cc:482] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.338346 17663 raft_consensus.cc:504] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.340171 17663 leader_election.cc:288] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.341811 17381 raft_consensus.cc:2454] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.344756 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.346544 17381 raft_consensus.cc:2379] T a732d904a75041c29d28ec6510e02a32 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:11.348062 17503 raft_consensus.cc:482] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.348562 17503 raft_consensus.cc:504] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.350548 17503 leader_election.cc:288] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.352113 17203 leader_election.cc:302] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.357136 17663 raft_consensus.cc:2781] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.357630 17663 raft_consensus.cc:686] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:11.358434 17663 consensus_queue.cc:237] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.360100 17427 tablet_bootstrap.cc:492] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.361198 17427 ts_tablet_manager.cc:1367] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.066s user 0.011s sys 0.004s
I20240501 08:04:11.364799 17427 raft_consensus.cc:348] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.368476 17503 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.369778 17503 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.370851 17427 raft_consensus.cc:374] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.371492 17427 raft_consensus.cc:729] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.371055 17426 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.372594 17426 ts_tablet_manager.cc:1367] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.052s user 0.015s sys 0.002s
I20240501 08:04:11.374368 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3708b02025940e589e0e97e994b0c70" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.375324 17427 consensus_queue.cc:260] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.377023 17427 ts_tablet_manager.cc:1398] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20240501 08:04:11.377440 17231 raft_consensus.cc:2454] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:11.380808 17427 tablet_bootstrap.cc:492] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.388262 17668 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.388759 17668 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.390974 17668 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.387755 17426 raft_consensus.cc:348] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.392223 17426 raft_consensus.cc:374] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.392534 17426 raft_consensus.cc:729] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.393249 17426 consensus_queue.cc:260] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.395606 17426 ts_tablet_manager.cc:1398] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20240501 08:04:11.396512 17426 tablet_bootstrap.cc:492] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.396561 17432 raft_consensus.cc:482] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.399711 17432 raft_consensus.cc:504] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.397318 17484 leader_election.cc:288] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.411880 17426 tablet_bootstrap.cc:654] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.406250 17663 raft_consensus.cc:482] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.415788 17432 leader_election.cc:288] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.420542 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3708b02025940e589e0e97e994b0c70" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.421018 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.421649 17381 raft_consensus.cc:2454] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.413878 17503 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.423245 17428 tablet_bootstrap.cc:492] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:11.421964 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.422808 17621 consensus_queue.cc:1035] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240501 08:04:11.425184 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.425772 17232 raft_consensus.cc:2454] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:11.426093 17428 ts_tablet_manager.cc:1367] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.244s user 0.036s sys 0.092s
I20240501 08:04:11.428752 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b47c5f05da46bc9d9b5b4b202c1210" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.429322 17311 raft_consensus.cc:3037] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.430064 17351 leader_election.cc:302] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.430855 17351 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.431026 17203 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.434098 17484 raft_consensus.cc:2781] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.436743 17427 tablet_bootstrap.cc:654] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.435678 17484 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.437180 17428 raft_consensus.cc:348] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.415064 17663 raft_consensus.cc:504] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.437899 17428 raft_consensus.cc:374] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.434671 17433 raft_consensus.cc:2781] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.436000 17503 raft_consensus.cc:2781] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.445526 17433 raft_consensus.cc:482] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.446005 17433 raft_consensus.cc:3037] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.445582 17503 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.446564 17503 raft_consensus.cc:3037] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.446903 17663 leader_election.cc:288] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:11.434136 17277 leader_election.cc:341] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.437713 17484 raft_consensus.cc:3037] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.450259 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732d904a75041c29d28ec6510e02a32" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.454077 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.454403 17668 raft_consensus.cc:482] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.454895 17668 raft_consensus.cc:504] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.505378 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726057013c1e48f985e1fd55e502c0e0" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.505275 17309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726057013c1e48f985e1fd55e502c0e0" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.486742 17311 raft_consensus.cc:2454] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.502378 17310 raft_consensus.cc:3037] T 38871d6cbe1846c89b09302d47294f87 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.447955 17428 raft_consensus.cc:729] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.494150 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44468f0ec3d94b97b8ea60830d3b7393" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.509402 17309 raft_consensus.cc:2454] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.488423 17503 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.496546 17663 consensus_queue.cc:1035] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20240501 08:04:11.511440 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44468f0ec3d94b97b8ea60830d3b7393" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.510751 17385 raft_consensus.cc:2454] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.512050 17307 raft_consensus.cc:2454] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.489110 17433 raft_consensus.cc:504] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.516491 17484 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.456707 17668 leader_election.cc:288] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:11.510690 17203 leader_election.cc:341] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.520175 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:11.520218 17203 leader_election.cc:302] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.518007 17433 leader_election.cc:288] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.521545 17668 raft_consensus.cc:2781] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.521350 17428 consensus_queue.cc:260] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.521886 17668 raft_consensus.cc:482] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.522238 17668 raft_consensus.cc:3037] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.520273 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3708b02025940e589e0e97e994b0c70" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:11.524520 17230 raft_consensus.cc:3037] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:11.528169 17351 leader_election.cc:341] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.530481 17202 leader_election.cc:302] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:11.531498 17428 ts_tablet_manager.cc:1398] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.105s user 0.025s sys 0.018s
W20240501 08:04:11.531594 17202 leader_election.cc:341] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.533191 17202 leader_election.cc:302] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:11.534875 17466 raft_consensus.cc:3037] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.535890 17667 raft_consensus.cc:2781] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.536254 17667 raft_consensus.cc:482] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.536564 17667 raft_consensus.cc:3037] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.538421 17426 tablet_bootstrap.cc:492] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.538841 17426 ts_tablet_manager.cc:1367] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.001s
I20240501 08:04:11.541887 17237 raft_consensus.cc:2379] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:11.542420 17466 raft_consensus.cc:2726] T a732d904a75041c29d28ec6510e02a32 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.544540 17484 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.545504 17667 raft_consensus.cc:504] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.545902 17432 raft_consensus.cc:482] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.546365 17432 raft_consensus.cc:504] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.548179 17432 leader_election.cc:288] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.551383 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.552434 17667 leader_election.cc:288] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.554318 17426 raft_consensus.cc:348] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.554991 17427 tablet_bootstrap.cc:492] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.555472 17427 ts_tablet_manager.cc:1367] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.175s user 0.014s sys 0.004s
I20240501 08:04:11.557224 17230 raft_consensus.cc:2454] T c3708b02025940e589e0e97e994b0c70 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:11.558205 17621 raft_consensus.cc:482] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.559265 17427 raft_consensus.cc:348] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.559947 17427 raft_consensus.cc:374] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.554993 17426 raft_consensus.cc:374] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.560583 17426 raft_consensus.cc:729] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.560204 17427 raft_consensus.cc:729] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.555536 17668 raft_consensus.cc:504] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.558640 17621 raft_consensus.cc:504] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.563784 17503 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.564060 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3708b02025940e589e0e97e994b0c70" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.562928 17427 consensus_queue.cc:260] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.566325 17426 consensus_queue.cc:260] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.564745 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.573231 17383 raft_consensus.cc:2379] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:11.574940 17428 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:11.575099 17621 leader_election.cc:288] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.564055 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.576812 17311 raft_consensus.cc:3037] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.582187 17351 leader_election.cc:302] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.578186 17382 raft_consensus.cc:3037] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.585911 17427 ts_tablet_manager.cc:1398] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.030s user 0.001s sys 0.005s
I20240501 08:04:11.586746 17427 tablet_bootstrap.cc:492] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.589665 17426 ts_tablet_manager.cc:1398] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.051s user 0.006s sys 0.001s
I20240501 08:04:11.594013 17426 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:11.587402 17504 raft_consensus.cc:2781] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.595237 17504 raft_consensus.cc:686] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:11.596051 17504 consensus_queue.cc:237] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.599969 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44468f0ec3d94b97b8ea60830d3b7393" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.604818 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.601763 17134 catalog_manager.cc:5484] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
W20240501 08:04:11.610347 17351 leader_election.cc:341] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.610459 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0274d08768ff4a1892e81b59d1677f2a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.610579 17619 consensus_queue.cc:1035] T 85c1e10c92f744c3a22dec7eecae1040 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.616639 17311 raft_consensus.cc:2454] T c3708b02025940e589e0e97e994b0c70 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:11.617504 17428 tablet_bootstrap.cc:654] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.613103 17668 leader_election.cc:288] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.623690 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0274d08768ff4a1892e81b59d1677f2a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.625536 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.628556 17427 tablet_bootstrap.cc:654] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:11.631016 17278 leader_election.cc:341] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.633416 17278 leader_election.cc:302] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.633558 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.634266 17432 raft_consensus.cc:2726] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.636628 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726057013c1e48f985e1fd55e502c0e0" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
W20240501 08:04:11.636817 17277 leader_election.cc:341] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.637192 17386 raft_consensus.cc:3037] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:11.637526 17277 leader_election.cc:341] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:04:11.637785 17203 leader_election.cc:341] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.641781 17616 raft_consensus.cc:482] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.641906 17621 consensus_queue.cc:1035] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.642222 17616 raft_consensus.cc:504] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:11.643769 17278 leader_election.cc:341] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.644201 17278 leader_election.cc:302] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.645545 17432 raft_consensus.cc:2726] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240501 08:04:11.653368 17278 leader_election.cc:341] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.653813 17278 leader_election.cc:302] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.652364 17616 leader_election.cc:288] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.654534 17488 raft_consensus.cc:2726] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.656832 17495 consensus_queue.cc:1035] T c7567a8d257c4c908317642193d10087 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.661360 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.633922 17426 tablet_bootstrap.cc:654] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.668093 17386 raft_consensus.cc:2454] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.669036 17133 catalog_manager.cc:5484] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:11.683571 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7567a8d257c4c908317642193d10087" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.687528 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44468f0ec3d94b97b8ea60830d3b7393" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.688164 17311 raft_consensus.cc:3037] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.693756 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726057013c1e48f985e1fd55e502c0e0" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.694504 17308 raft_consensus.cc:3037] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.705153 17203 leader_election.cc:302] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.706823 17668 raft_consensus.cc:2781] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.707378 17668 raft_consensus.cc:686] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:11.708243 17668 consensus_queue.cc:237] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:11.716071 17351 leader_election.cc:341] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.716621 17351 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.717792 17433 raft_consensus.cc:2726] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.723276 17484 consensus_queue.cc:1035] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20240501 08:04:11.723322 17202 leader_election.cc:341] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.725199 17202 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:11.726351 17667 raft_consensus.cc:2726] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:04:11.747012 17432 consensus_queue.cc:1035] T a732d904a75041c29d28ec6510e02a32 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:11.746250 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7567a8d257c4c908317642193d10087" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.771023 17427 tablet_bootstrap.cc:492] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.771515 17427 ts_tablet_manager.cc:1367] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.185s user 0.018s sys 0.021s
I20240501 08:04:11.773814 17427 raft_consensus.cc:348] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.774502 17427 raft_consensus.cc:374] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.774755 17427 raft_consensus.cc:729] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.775312 17427 consensus_queue.cc:260] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.775852 17351 leader_election.cc:302] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:11.776831 17427 ts_tablet_manager.cc:1398] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 08:04:11.777655 17427 tablet_bootstrap.cc:492] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:11.788987 17311 raft_consensus.cc:2454] T 44468f0ec3d94b97b8ea60830d3b7393 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.790230 17202 leader_election.cc:302] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:11.791365 17619 raft_consensus.cc:2781] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.791831 17619 raft_consensus.cc:686] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:11.792709 17308 raft_consensus.cc:2454] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.793126 17619 consensus_queue.cc:237] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:11.793685 17203 consensus_peers.cc:485] T b993df33624a4500a73b663ca80d75f7 P fa9d3bc74a68481e9de647223cb0110e -> Peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Couldn't send request to peer 5655c2e9168d4eeaa212db8874c2efa5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:11.794454 17667 raft_consensus.cc:482] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.794879 17667 raft_consensus.cc:504] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.800292 17667 leader_election.cc:288] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.801674 17428 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:11.802076 17428 ts_tablet_manager.cc:1367] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.227s user 0.044s sys 0.034s
W20240501 08:04:11.816603 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:11.807783 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.817770 17306 raft_consensus.cc:2454] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:11.818472 17428 raft_consensus.cc:348] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.819264 17428 raft_consensus.cc:374] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.820003 17202 leader_election.cc:302] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.821156 17428 raft_consensus.cc:729] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.821918 17428 consensus_queue.cc:260] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.824604 17503 raft_consensus.cc:482] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:11.825484 17466 raft_consensus.cc:2781] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.826050 17466 raft_consensus.cc:482] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.825716 17503 raft_consensus.cc:504] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.826407 17466 raft_consensus.cc:3037] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.829835 17616 raft_consensus.cc:2726] T c7567a8d257c4c908317642193d10087 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240501 08:04:11.830832 17428 ts_tablet_manager.cc:1398] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.028s user 0.008s sys 0.000s
I20240501 08:04:11.831697 17428 tablet_bootstrap.cc:492] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:11.827395 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:11.832971 17427 tablet_bootstrap.cc:654] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:11.833446 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990374acc0c34c788d07bca6603f2e1e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:11.839262 17503 leader_election.cc:288] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.834059 17232 raft_consensus.cc:2454] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:11.841904 17351 leader_election.cc:302] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.849102 17504 raft_consensus.cc:2781] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:11.855603 17504 raft_consensus.cc:482] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:11.855990 17504 raft_consensus.cc:3037] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.859104 17466 raft_consensus.cc:504] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.861197 17466 leader_election.cc:288] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:11.924976 17426 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:11.925465 17426 ts_tablet_manager.cc:1367] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.332s user 0.017s sys 0.019s
W20240501 08:04:11.928966 17203 leader_election.cc:341] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.924453 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990374acc0c34c788d07bca6603f2e1e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:11.934072 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:11.929392 17504 raft_consensus.cc:504] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.940387 17504 leader_election.cc:288] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:11.935439 17305 raft_consensus.cc:3037] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.943661 17428 tablet_bootstrap.cc:654] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:11.944975 17351 leader_election.cc:341] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.949456 17619 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:11.953708 17134 catalog_manager.cc:5484] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:11.955704 17134 catalog_manager.cc:5484] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:11.953699 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990374acc0c34c788d07bca6603f2e1e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:11.961020 17230 raft_consensus.cc:3037] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:11.968348 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:11.970788 17230 raft_consensus.cc:2454] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:11.972584 17426 raft_consensus.cc:348] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.973891 17351 leader_election.cc:302] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.974624 17305 raft_consensus.cc:2454] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:11.976326 17426 raft_consensus.cc:374] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:11.977712 17426 raft_consensus.cc:729] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:11.978293 17426 consensus_queue.cc:260] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.978284 17504 raft_consensus.cc:2781] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.984150 17504 raft_consensus.cc:686] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:11.984990 17504 consensus_queue.cc:237] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.987951 17202 leader_election.cc:302] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:11.988734 17466 raft_consensus.cc:2781] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:11.989202 17466 raft_consensus.cc:686] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:11.990061 17466 consensus_queue.cc:237] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:11.978531 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990374acc0c34c788d07bca6603f2e1e" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
W20240501 08:04:11.992146 17351 leader_election.cc:341] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:11.994402 17427 tablet_bootstrap.cc:492] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:11.995227 17427 ts_tablet_manager.cc:1367] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.218s user 0.028s sys 0.022s
I20240501 08:04:12.002279 17426 ts_tablet_manager.cc:1398] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.077s user 0.006s sys 0.000s
I20240501 08:04:12.004201 17426 tablet_bootstrap.cc:492] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
W20240501 08:04:12.010073 17203 leader_election.cc:341] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.011334 17619 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:12.011308 17427 raft_consensus.cc:348] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.012207 17427 raft_consensus.cc:374] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.012471 17427 raft_consensus.cc:729] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.013329 17503 raft_consensus.cc:482] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.013898 17503 raft_consensus.cc:504] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.015426 17495 raft_consensus.cc:482] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.013376 17427 consensus_queue.cc:260] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.018533 17503 leader_election.cc:288] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:12.015853 17495 raft_consensus.cc:504] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.021224 17495 leader_election.cc:288] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.023789 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.024011 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66624ff54d8844e3bf88af88be895fd3" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
W20240501 08:04:12.026645 17351 leader_election.cc:341] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:04:12.027936 17277 leader_election.cc:341] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.029457 17427 ts_tablet_manager.cc:1398] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20240501 08:04:12.030442 17427 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.033179 17134 catalog_manager.cc:5484] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:12.034209 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.035120 17307 raft_consensus.cc:2454] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:12.036109 17351 leader_election.cc:302] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.037046 17503 raft_consensus.cc:2781] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:12.037412 17503 raft_consensus.cc:482] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:12.037717 17503 raft_consensus.cc:3037] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.040323 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66624ff54d8844e3bf88af88be895fd3" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.055588 17133 catalog_manager.cc:5484] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:12.062665 17426 tablet_bootstrap.cc:654] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:12.075918 17278 leader_election.cc:341] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.076254 17278 leader_election.cc:302] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.069334 17503 raft_consensus.cc:504] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.077030 17432 raft_consensus.cc:2726] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.079073 17427 tablet_bootstrap.cc:654] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.090440 17503 leader_election.cc:288] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:12.092494 17428 tablet_bootstrap.cc:492] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.092950 17428 ts_tablet_manager.cc:1367] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.261s user 0.012s sys 0.003s
I20240501 08:04:12.094645 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
W20240501 08:04:12.096045 17351 leader_election.cc:341] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.094712 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:12.097443 17306 raft_consensus.cc:3037] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.097882 17428 raft_consensus.cc:348] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.098555 17428 raft_consensus.cc:374] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.098807 17428 raft_consensus.cc:729] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.099416 17428 consensus_queue.cc:260] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.104478 17306 raft_consensus.cc:2454] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:12.106523 17351 leader_election.cc:302] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.107117 17503 raft_consensus.cc:2781] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:12.107573 17503 raft_consensus.cc:686] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:12.108311 17503 consensus_queue.cc:237] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.113006 17427 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.113159 17428 ts_tablet_manager.cc:1398] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.020s user 0.004s sys 0.004s
I20240501 08:04:12.113440 17427 ts_tablet_manager.cc:1367] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.083s user 0.010s sys 0.000s
I20240501 08:04:12.114109 17428 tablet_bootstrap.cc:492] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.114748 17426 tablet_bootstrap.cc:492] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.115211 17426 ts_tablet_manager.cc:1367] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.111s user 0.014s sys 0.000s
I20240501 08:04:12.115809 17427 raft_consensus.cc:348] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.116498 17427 raft_consensus.cc:374] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.116753 17427 raft_consensus.cc:729] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.117349 17427 consensus_queue.cc:260] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.117444 17426 raft_consensus.cc:348] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.118352 17426 raft_consensus.cc:374] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.118609 17426 raft_consensus.cc:729] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.120110 17426 consensus_queue.cc:260] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.125886 17426 ts_tablet_manager.cc:1398] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240501 08:04:12.126367 17427 ts_tablet_manager.cc:1398] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240501 08:04:12.126793 17426 tablet_bootstrap.cc:492] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.127192 17427 tablet_bootstrap.cc:492] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.126698 17133 catalog_manager.cc:5484] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:12.132579 17426 tablet_bootstrap.cc:654] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.133013 17427 tablet_bootstrap.cc:654] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.118696 17428 tablet_bootstrap.cc:654] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.154096 17663 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.161166 17504 consensus_queue.cc:1035] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.170207 17428 tablet_bootstrap.cc:492] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.170733 17428 ts_tablet_manager.cc:1367] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.057s user 0.032s sys 0.008s
I20240501 08:04:12.173170 17428 raft_consensus.cc:348] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.174552 17503 consensus_queue.cc:1035] T c3708b02025940e589e0e97e994b0c70 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.173872 17428 raft_consensus.cc:374] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.175552 17428 raft_consensus.cc:729] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.176136 17428 consensus_queue.cc:260] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.178217 17428 ts_tablet_manager.cc:1398] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 08:04:12.179194 17428 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.187467 17386 raft_consensus.cc:3037] T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.187330 17427 tablet_bootstrap.cc:492] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.188297 17427 ts_tablet_manager.cc:1367] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.061s user 0.027s sys 0.001s
I20240501 08:04:12.192669 17427 raft_consensus.cc:348] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.193477 17427 raft_consensus.cc:374] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.193796 17427 raft_consensus.cc:729] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.194850 17428 tablet_bootstrap.cc:654] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.194406 17427 consensus_queue.cc:260] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.201465 17427 ts_tablet_manager.cc:1398] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.013s user 0.004s sys 0.002s
I20240501 08:04:12.202265 17427 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.204550 17426 tablet_bootstrap.cc:492] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.205013 17426 ts_tablet_manager.cc:1367] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.078s user 0.017s sys 0.001s
I20240501 08:04:12.215905 17466 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:12.225937 17427 tablet_bootstrap.cc:654] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.230645 17426 raft_consensus.cc:348] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.232487 17426 raft_consensus.cc:374] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.232841 17426 raft_consensus.cc:729] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.264827 17426 consensus_queue.cc:260] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.266752 17426 ts_tablet_manager.cc:1398] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.061s user 0.015s sys 0.013s
I20240501 08:04:12.267670 17426 tablet_bootstrap.cc:492] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.273701 17426 tablet_bootstrap.cc:654] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.279289 17466 raft_consensus.cc:482] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.280346 17428 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.280267 17466 raft_consensus.cc:504] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.280794 17428 ts_tablet_manager.cc:1367] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.102s user 0.010s sys 0.013s
I20240501 08:04:12.283170 17428 raft_consensus.cc:348] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.283823 17428 raft_consensus.cc:374] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.284080 17428 raft_consensus.cc:729] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.284689 17428 consensus_queue.cc:260] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.286140 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4437b60cbdcd4b7d850f3534a8845e92" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.286923 17428 ts_tablet_manager.cc:1398] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20240501 08:04:12.287242 17466 leader_election.cc:288] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:12.287825 17202 leader_election.cc:341] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.286741 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4437b60cbdcd4b7d850f3534a8845e92" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.288256 17428 tablet_bootstrap.cc:492] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
W20240501 08:04:12.289203 17203 leader_election.cc:341] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.289630 17203 leader_election.cc:302] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:12.290508 17466 raft_consensus.cc:2726] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.293944 17428 tablet_bootstrap.cc:654] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:12.295994 17706 log.cc:924] Time spent T b993df33624a4500a73b663ca80d75f7 P 5655c2e9168d4eeaa212db8874c2efa5: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20240501 08:04:12.309877 17504 raft_consensus.cc:482] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.310348 17504 raft_consensus.cc:504] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.312019 17504 leader_election.cc:288] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:12.313014 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.316108 17428 tablet_bootstrap.cc:492] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.316558 17428 ts_tablet_manager.cc:1367] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.001s
I20240501 08:04:12.316712 17663 consensus_queue.cc:1035] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.336833 17432 raft_consensus.cc:482] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.337433 17432 raft_consensus.cc:504] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.361701 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a7c27c8f0b41dca4f760d78b7d94fc" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.362052 17432 leader_election.cc:288] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.383913 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990374acc0c34c788d07bca6603f2e1e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.408591 17307 raft_consensus.cc:2379] T 64a7c27c8f0b41dca4f760d78b7d94fc P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:12.428360 17432 raft_consensus.cc:482] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.440804 17432 raft_consensus.cc:504] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.442750 17432 leader_election.cc:288] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.443310 17428 raft_consensus.cc:348] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.443907 17351 leader_election.cc:302] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.444141 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.444624 17433 raft_consensus.cc:3037] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.444051 17428 raft_consensus.cc:374] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.445112 17428 raft_consensus.cc:729] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
W20240501 08:04:12.445663 17278 leader_election.cc:341] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.445767 17428 consensus_queue.cc:260] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.447258 17428 ts_tablet_manager.cc:1398] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.130s user 0.004s sys 0.004s
I20240501 08:04:12.448246 17428 tablet_bootstrap.cc:492] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.454704 17663 raft_consensus.cc:482] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.455228 17663 raft_consensus.cc:504] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.457594 17433 raft_consensus.cc:2726] T 64a7c27c8f0b41dca4f760d78b7d94fc P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.459724 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0274d08768ff4a1892e81b59d1677f2a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.460363 17306 raft_consensus.cc:2454] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:12.457165 17663 leader_election.cc:288] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.462109 17202 leader_election.cc:302] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:12.463155 17484 raft_consensus.cc:2781] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:12.463591 17484 raft_consensus.cc:482] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:12.463912 17484 raft_consensus.cc:3037] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.466341 17428 tablet_bootstrap.cc:654] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.473155 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990374acc0c34c788d07bca6603f2e1e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.475778 17232 raft_consensus.cc:2379] T 990374acc0c34c788d07bca6603f2e1e P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:12.478385 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.479588 17484 raft_consensus.cc:504] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.480294 17426 tablet_bootstrap.cc:492] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.480746 17426 ts_tablet_manager.cc:1367] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.213s user 0.013s sys 0.008s
I20240501 08:04:12.481102 17277 leader_election.cc:302] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.481842 17495 raft_consensus.cc:3037] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.483124 17426 raft_consensus.cc:348] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.483726 17426 raft_consensus.cc:374] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240501 08:04:12.484447 17277 leader_election.cc:341] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.484839 17277 leader_election.cc:302] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.486220 17621 raft_consensus.cc:2726] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.483932 17426 raft_consensus.cc:729] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.514526 17427 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.533334 17427 ts_tablet_manager.cc:1367] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.331s user 0.020s sys 0.023s
I20240501 08:04:12.533187 17426 consensus_queue.cc:260] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.537616 17484 leader_election.cc:288] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.539433 17426 ts_tablet_manager.cc:1398] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.058s user 0.009s sys 0.016s
I20240501 08:04:12.540567 17495 raft_consensus.cc:2726] T 990374acc0c34c788d07bca6603f2e1e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.541083 17704 consensus_queue.cc:1035] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.541951 17426 tablet_bootstrap.cc:492] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.555233 17621 raft_consensus.cc:482] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.555747 17621 raft_consensus.cc:504] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.556125 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0274d08768ff4a1892e81b59d1677f2a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:12.556486 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0274d08768ff4a1892e81b59d1677f2a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:12.556720 17381 raft_consensus.cc:3037] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.557101 17305 raft_consensus.cc:3037] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.557811 17621 leader_election.cc:288] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:12.560765 17203 consensus_peers.cc:485] T 44468f0ec3d94b97b8ea60830d3b7393 P fa9d3bc74a68481e9de647223cb0110e -> Peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Couldn't send request to peer 5655c2e9168d4eeaa212db8874c2efa5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:12.561681 17427 raft_consensus.cc:348] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.562412 17427 raft_consensus.cc:374] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.562700 17427 raft_consensus.cc:729] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.563289 17427 consensus_queue.cc:260] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.568570 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0274d08768ff4a1892e81b59d1677f2a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.572185 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4437b60cbdcd4b7d850f3534a8845e92" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.573439 17427 ts_tablet_manager.cc:1398] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.037s user 0.007s sys 0.000s
I20240501 08:04:12.574585 17427 tablet_bootstrap.cc:492] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.575593 17428 tablet_bootstrap.cc:492] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.576009 17428 ts_tablet_manager.cc:1367] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.128s user 0.013s sys 0.001s
I20240501 08:04:12.573263 17231 raft_consensus.cc:2454] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:12.573426 17667 consensus_queue.cc:1035] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20240501 08:04:12.576573 17504 consensus_queue.cc:1035] T 990374acc0c34c788d07bca6603f2e1e P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:12.581718 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4437b60cbdcd4b7d850f3534a8845e92" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.579895 17428 raft_consensus.cc:348] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.584715 17277 leader_election.cc:302] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:12.585098 17428 raft_consensus.cc:374] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.585506 17428 raft_consensus.cc:729] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.587934 17428 consensus_queue.cc:260] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.589339 17714 raft_consensus.cc:2781] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:12.591519 17714 raft_consensus.cc:482] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:12.591837 17714 raft_consensus.cc:3037] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.596575 17428 ts_tablet_manager.cc:1398] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.020s user 0.003s sys 0.003s
I20240501 08:04:12.597468 17428 tablet_bootstrap.cc:492] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.597748 17426 tablet_bootstrap.cc:654] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.588287 17667 raft_consensus.cc:482] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.601222 17667 raft_consensus.cc:504] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.600387 17305 raft_consensus.cc:2454] T 0274d08768ff4a1892e81b59d1677f2a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:12.612680 17428 tablet_bootstrap.cc:654] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.611966 17231 raft_consensus.cc:3037] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.607462 17714 raft_consensus.cc:504] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.621575 17427 tablet_bootstrap.cc:654] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.627569 17426 tablet_bootstrap.cc:492] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.627617 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4437b60cbdcd4b7d850f3534a8845e92" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:12.628378 17426 ts_tablet_manager.cc:1367] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.012s
I20240501 08:04:12.628538 17237 raft_consensus.cc:3037] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.630882 17426 raft_consensus.cc:348] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.631646 17426 raft_consensus.cc:374] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.631942 17426 raft_consensus.cc:729] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.632863 17713 raft_consensus.cc:482] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.633303 17713 raft_consensus.cc:504] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.635087 17713 leader_election.cc:288] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:12.635882 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66624ff54d8844e3bf88af88be895fd3" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.636500 17312 raft_consensus.cc:2454] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:12.632570 17426 consensus_queue.cc:260] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:12.637562 17278 leader_election.cc:341] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.638772 17351 leader_election.cc:302] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:12.642357 17426 ts_tablet_manager.cc:1398] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20240501 08:04:12.643306 17426 tablet_bootstrap.cc:492] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.639997 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66624ff54d8844e3bf88af88be895fd3" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.640761 17713 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:12.642369 17704 raft_consensus.cc:2781] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:12.645278 17714 leader_election.cc:288] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.651568 17704 raft_consensus.cc:482] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:12.652091 17230 raft_consensus.cc:2454] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:12.656652 17202 leader_election.cc:302] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:12.663282 17484 raft_consensus.cc:2781] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:12.663883 17484 raft_consensus.cc:686] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:12.664358 17663 consensus_queue.cc:1035] T 64a7c27c8f0b41dca4f760d78b7d94fc P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.664646 17484 consensus_queue.cc:237] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.666121 17712 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:12.666555 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.660454 17704 raft_consensus.cc:3037] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.667294 17307 raft_consensus.cc:2379] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa9d3bc74a68481e9de647223cb0110e in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:12.668671 17381 raft_consensus.cc:2454] T 0274d08768ff4a1892e81b59d1677f2a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:12.678968 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4437b60cbdcd4b7d850f3534a8845e92" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:12.662220 17667 leader_election.cc:288] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.682580 17712 raft_consensus.cc:482] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240501 08:04:12.688234 17278 leader_election.cc:341] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.688400 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f504d6ad6b4b77927fb216e12347e8" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.685449 17712 raft_consensus.cc:504] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.684264 17237 raft_consensus.cc:2454] T 4437b60cbdcd4b7d850f3534a8845e92 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:12.701577 17704 raft_consensus.cc:504] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.704061 17712 leader_election.cc:288] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:12.706013 17704 leader_election.cc:288] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:12.707144 17426 tablet_bootstrap.cc:654] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.709431 17277 leader_election.cc:302] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:12.710175 17432 raft_consensus.cc:2781] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:12.710621 17432 raft_consensus.cc:686] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:12.712476 17203 leader_election.cc:302] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:12.719962 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44468f0ec3d94b97b8ea60830d3b7393" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.720219 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66624ff54d8844e3bf88af88be895fd3" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:12.720741 17307 raft_consensus.cc:3037] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.720968 17427 tablet_bootstrap.cc:492] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.714643 17667 raft_consensus.cc:2726] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.721477 17427 ts_tablet_manager.cc:1367] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.147s user 0.014s sys 0.008s
I20240501 08:04:12.722599 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44468f0ec3d94b97b8ea60830d3b7393" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:12.711395 17432 consensus_queue.cc:237] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.725987 17428 tablet_bootstrap.cc:492] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.726518 17428 ts_tablet_manager.cc:1367] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.129s user 0.006s sys 0.005s
I20240501 08:04:12.720702 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66624ff54d8844e3bf88af88be895fd3" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:12.727881 17231 raft_consensus.cc:3037] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.733670 17427 raft_consensus.cc:348] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.734375 17427 raft_consensus.cc:374] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.735632 17428 raft_consensus.cc:348] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.736516 17428 raft_consensus.cc:374] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.736992 17428 raft_consensus.cc:729] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.737679 17428 consensus_queue.cc:260] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.738144 17132 catalog_manager.cc:5484] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:12.740339 17428 ts_tablet_manager.cc:1398] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240501 08:04:12.741202 17428 tablet_bootstrap.cc:492] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.742306 17427 raft_consensus.cc:729] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.742986 17427 consensus_queue.cc:260] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.746667 17427 ts_tablet_manager.cc:1398] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20240501 08:04:12.749104 17427 tablet_bootstrap.cc:492] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.744087 17231 raft_consensus.cc:2454] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:12.767251 17351 leader_election.cc:302] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:12.768335 17351 leader_election.cc:302] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:12.769073 17712 raft_consensus.cc:2726] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:12.769665 17712 raft_consensus.cc:2781] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:12.770092 17712 raft_consensus.cc:686] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:12.770870 17712 consensus_queue.cc:237] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.773536 17427 tablet_bootstrap.cc:654] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.781364 17307 raft_consensus.cc:2454] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:12.823035 17428 tablet_bootstrap.cc:654] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.826016 17134 catalog_manager.cc:5484] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:12.855625 17426 tablet_bootstrap.cc:492] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.856174 17426 ts_tablet_manager.cc:1367] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.213s user 0.017s sys 0.008s
I20240501 08:04:12.873347 17426 raft_consensus.cc:348] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.876327 17426 raft_consensus.cc:374] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.876686 17426 raft_consensus.cc:729] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.877979 17426 consensus_queue.cc:260] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.881119 17426 ts_tablet_manager.cc:1398] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.025s user 0.003s sys 0.003s
I20240501 08:04:12.882414 17426 tablet_bootstrap.cc:492] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.884768 17428 tablet_bootstrap.cc:492] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.885390 17428 ts_tablet_manager.cc:1367] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.144s user 0.004s sys 0.025s
I20240501 08:04:12.889010 17428 raft_consensus.cc:348] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.889904 17428 raft_consensus.cc:374] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.890223 17428 raft_consensus.cc:729] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.890616 17426 tablet_bootstrap.cc:654] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.891006 17428 consensus_queue.cc:260] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.893707 17428 ts_tablet_manager.cc:1398] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240501 08:04:12.895002 17427 tablet_bootstrap.cc:492] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.895942 17427 ts_tablet_manager.cc:1367] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.149s user 0.023s sys 0.000s
I20240501 08:04:12.896780 17428 tablet_bootstrap.cc:492] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.899093 17427 raft_consensus.cc:348] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.899820 17427 raft_consensus.cc:374] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.900110 17427 raft_consensus.cc:729] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.900740 17427 consensus_queue.cc:260] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.903131 17427 ts_tablet_manager.cc:1398] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 08:04:12.904067 17427 tablet_bootstrap.cc:492] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.910033 17427 tablet_bootstrap.cc:654] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.910919 17428 tablet_bootstrap.cc:654] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.913079 17426 tablet_bootstrap.cc:492] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.913532 17426 ts_tablet_manager.cc:1367] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.005s
I20240501 08:04:12.915890 17426 raft_consensus.cc:348] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.916393 17426 raft_consensus.cc:374] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.916595 17426 raft_consensus.cc:729] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.917090 17426 consensus_queue.cc:260] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.918578 17426 ts_tablet_manager.cc:1398] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 08:04:12.919242 17426 tablet_bootstrap.cc:492] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.924185 17426 tablet_bootstrap.cc:654] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.927095 17427 tablet_bootstrap.cc:492] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.927577 17427 ts_tablet_manager.cc:1367] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.000s
I20240501 08:04:12.928588 17428 tablet_bootstrap.cc:492] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.929016 17428 ts_tablet_manager.cc:1367] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.009s
I20240501 08:04:12.930307 17427 raft_consensus.cc:348] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.931113 17427 raft_consensus.cc:374] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.931180 17428 raft_consensus.cc:348] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.931556 17427 raft_consensus.cc:729] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.931890 17428 raft_consensus.cc:374] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.932394 17427 consensus_queue.cc:260] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.932196 17428 raft_consensus.cc:729] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.933987 17428 consensus_queue.cc:260] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.934597 17427 ts_tablet_manager.cc:1398] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20240501 08:04:12.935518 17427 tablet_bootstrap.cc:492] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.936359 17484 raft_consensus.cc:482] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:12.936851 17484 raft_consensus.cc:504] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.938733 17484 leader_election.cc:288] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.940104 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:12.940711 17428 ts_tablet_manager.cc:1398] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
I20240501 08:04:12.941571 17428 tablet_bootstrap.cc:492] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
W20240501 08:04:12.941475 17203 leader_election.cc:341] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.942198 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:12.942819 17312 raft_consensus.cc:2454] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:12.944867 17427 tablet_bootstrap.cc:654] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.945399 17202 leader_election.cc:302] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:12.946813 17663 raft_consensus.cc:2781] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:12.947155 17663 raft_consensus.cc:482] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:12.947482 17663 raft_consensus.cc:3037] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.947688 17428 tablet_bootstrap.cc:654] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:12.949828 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:12.951925 17426 tablet_bootstrap.cc:492] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:12.952656 17426 ts_tablet_manager.cc:1367] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.007s
I20240501 08:04:12.954475 17381 raft_consensus.cc:3037] T 44468f0ec3d94b97b8ea60830d3b7393 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.955066 17426 raft_consensus.cc:348] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.955794 17426 raft_consensus.cc:374] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.956084 17426 raft_consensus.cc:729] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.956719 17426 consensus_queue.cc:260] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.958979 17426 ts_tablet_manager.cc:1398] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:12.959894 17426 tablet_bootstrap.cc:492] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:12.962373 17663 raft_consensus.cc:504] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.969573 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
W20240501 08:04:12.971539 17203 leader_election.cc:341] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:12.974725 17132 catalog_manager.cc:5484] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:12.976241 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:12.976801 17312 raft_consensus.cc:3037] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:12.972029 17663 leader_election.cc:288] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:12.983196 17428 tablet_bootstrap.cc:492] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:12.983633 17428 ts_tablet_manager.cc:1367] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.042s user 0.013s sys 0.006s
I20240501 08:04:12.985431 17312 raft_consensus.cc:2454] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:12.986471 17426 tablet_bootstrap.cc:654] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:12.986008 17428 raft_consensus.cc:348] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.986755 17428 raft_consensus.cc:374] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.987017 17428 raft_consensus.cc:729] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.987088 17202 leader_election.cc:302] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:12.987594 17428 consensus_queue.cc:260] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.989629 17427 tablet_bootstrap.cc:492] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:12.990057 17427 ts_tablet_manager.cc:1367] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.055s user 0.017s sys 0.004s
I20240501 08:04:12.990177 17484 raft_consensus.cc:2781] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:12.990707 17484 raft_consensus.cc:686] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:12.991474 17484 consensus_queue.cc:237] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.990186 17428 ts_tablet_manager.cc:1398] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20240501 08:04:12.992353 17427 raft_consensus.cc:348] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.993084 17427 raft_consensus.cc:374] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:12.993363 17428 tablet_bootstrap.cc:492] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:12.993371 17427 raft_consensus.cc:729] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:12.994158 17427 consensus_queue.cc:260] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:12.996099 17427 ts_tablet_manager.cc:1398] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240501 08:04:12.996933 17427 tablet_bootstrap.cc:492] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:12.998710 17428 tablet_bootstrap.cc:654] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.001787 17427 tablet_bootstrap.cc:654] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.026917 17426 tablet_bootstrap.cc:492] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.027367 17426 ts_tablet_manager.cc:1367] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.008s
I20240501 08:04:13.034848 17428 tablet_bootstrap.cc:492] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:13.035446 17428 ts_tablet_manager.cc:1367] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.042s user 0.019s sys 0.009s
I20240501 08:04:13.038028 17428 raft_consensus.cc:348] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.038754 17428 raft_consensus.cc:374] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.039058 17428 raft_consensus.cc:729] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.039026 17426 raft_consensus.cc:348] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.039825 17426 raft_consensus.cc:374] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.040107 17426 raft_consensus.cc:729] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.040381 17427 tablet_bootstrap.cc:492] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.040205 17428 consensus_queue.cc:260] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.039525 17132 catalog_manager.cc:5484] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:13.040858 17427 ts_tablet_manager.cc:1367] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.044s user 0.026s sys 0.011s
I20240501 08:04:13.040823 17426 consensus_queue.cc:260] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.042882 17428 ts_tablet_manager.cc:1398] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20240501 08:04:13.042887 17426 ts_tablet_manager.cc:1398] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20240501 08:04:13.043771 17428 tablet_bootstrap.cc:492] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:13.044622 17426 tablet_bootstrap.cc:492] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.050330 17427 raft_consensus.cc:348] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.050879 17427 raft_consensus.cc:374] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.051072 17427 raft_consensus.cc:729] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.055250 17427 consensus_queue.cc:260] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.057268 17427 ts_tablet_manager.cc:1398] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.016s user 0.000s sys 0.003s
I20240501 08:04:13.061975 17427 tablet_bootstrap.cc:492] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.064139 17426 tablet_bootstrap.cc:654] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.059406 17428 tablet_bootstrap.cc:654] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.066924 17427 tablet_bootstrap.cc:654] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.066905 17713 raft_consensus.cc:482] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.067554 17713 raft_consensus.cc:504] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.070048 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867eb9d6d70b41ddb63cd0e79921d391" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.070607 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867eb9d6d70b41ddb63cd0e79921d391" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.070747 17232 raft_consensus.cc:2454] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:13.071892 17351 leader_election.cc:302] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.072650 17712 raft_consensus.cc:2781] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.073032 17712 raft_consensus.cc:482] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20240501 08:04:13.072733 17351 leader_election.cc:341] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.073019 17713 leader_election.cc:288] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.073662 17712 raft_consensus.cc:3037] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.076398 17428 tablet_bootstrap.cc:492] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:13.076844 17428 ts_tablet_manager.cc:1367] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.003s
I20240501 08:04:13.079149 17428 raft_consensus.cc:348] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.079790 17428 raft_consensus.cc:374] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.080030 17428 raft_consensus.cc:729] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.080582 17428 consensus_queue.cc:260] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.082401 17428 ts_tablet_manager.cc:1398] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20240501 08:04:13.084523 17426 tablet_bootstrap.cc:492] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.084959 17426 ts_tablet_manager.cc:1367] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.009s
I20240501 08:04:13.085150 17712 raft_consensus.cc:504] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.086884 17428 tablet_bootstrap.cc:492] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:13.087913 17427 tablet_bootstrap.cc:492] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.088379 17427 ts_tablet_manager.cc:1367] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.001s
I20240501 08:04:13.089044 17714 raft_consensus.cc:482] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.089545 17714 raft_consensus.cc:504] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.090034 17432 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.090580 17432 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.091363 17714 leader_election.cc:288] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.091719 17712 leader_election.cc:288] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.092761 17432 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.093386 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8945347fd484e038a72f8580fc88472" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.094064 17384 raft_consensus.cc:2454] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:13.094908 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.095141 17278 leader_election.cc:302] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:13.095638 17381 raft_consensus.cc:2379] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate 5655c2e9168d4eeaa212db8874c2efa5 in this term.
I20240501 08:04:13.095942 17428 tablet_bootstrap.cc:654] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.092666 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867eb9d6d70b41ddb63cd0e79921d391" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.093467 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867eb9d6d70b41ddb63cd0e79921d391" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:13.096693 17232 raft_consensus.cc:3037] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.096848 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8945347fd484e038a72f8580fc88472" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.096278 17432 raft_consensus.cc:2781] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.098297 17432 raft_consensus.cc:482] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.098369 17427 raft_consensus.cc:348] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.099227 17427 raft_consensus.cc:374] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.098757 17432 raft_consensus.cc:3037] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.099552 17427 raft_consensus.cc:729] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
W20240501 08:04:13.100834 17351 leader_election.cc:341] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.100833 17427 consensus_queue.cc:260] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.102027 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.102878 17230 raft_consensus.cc:2379] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:13.103561 17427 ts_tablet_manager.cc:1398] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
W20240501 08:04:13.104166 17277 leader_election.cc:341] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.104458 17427 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.105229 17713 raft_consensus.cc:482] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.105687 17713 raft_consensus.cc:504] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.107453 17713 leader_election.cc:288] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.107859 17713 raft_consensus.cc:482] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.108230 17713 raft_consensus.cc:504] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.109139 17232 raft_consensus.cc:2454] T 867eb9d6d70b41ddb63cd0e79921d391 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:13.109475 17277 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.110563 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.111784 17714 raft_consensus.cc:3037] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:13.112136 17351 leader_election.cc:341] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.118008 17351 leader_election.cc:302] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.118345 17427 tablet_bootstrap.cc:654] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.118500 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.118717 17712 raft_consensus.cc:2781] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.119215 17712 raft_consensus.cc:686] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:13.120257 17713 leader_election.cc:288] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.119964 17712 consensus_queue.cc:237] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:13.123173 17351 leader_election.cc:341] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.123616 17351 leader_election.cc:302] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.124755 17713 raft_consensus.cc:2726] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.120508 17714 raft_consensus.cc:2726] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.121851 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa513fcceeb4a15b0a193ddf00a6e62" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.125888 17307 raft_consensus.cc:2454] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:13.089020 17426 raft_consensus.cc:348] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.127244 17426 raft_consensus.cc:374] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.127056 17351 leader_election.cc:302] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters:
I20240501 08:04:13.127581 17426 raft_consensus.cc:729] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.126014 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa513fcceeb4a15b0a193ddf00a6e62" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.136091 17230 raft_consensus.cc:2454] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:13.136271 17426 consensus_queue.cc:260] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.137645 17713 raft_consensus.cc:2781] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.140339 17426 ts_tablet_manager.cc:1398] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.055s user 0.007s sys 0.002s
I20240501 08:04:13.141714 17426 tablet_bootstrap.cc:492] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.138945 17713 raft_consensus.cc:482] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.144752 17713 raft_consensus.cc:3037] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.144204 17432 raft_consensus.cc:504] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.149744 17432 leader_election.cc:288] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.150310 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8945347fd484e038a72f8580fc88472" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:13.150940 17381 raft_consensus.cc:3037] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.156822 17426 tablet_bootstrap.cc:654] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.165549 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8945347fd484e038a72f8580fc88472" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:13.166049 17132 catalog_manager.cc:5484] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
W20240501 08:04:13.167482 17277 leader_election.cc:341] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.173214 17713 raft_consensus.cc:504] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.175357 17381 raft_consensus.cc:2454] T a8945347fd484e038a72f8580fc88472 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:13.181707 17428 tablet_bootstrap.cc:492] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:13.183866 17714 consensus_queue.cc:1035] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:13.202093 17621 raft_consensus.cc:482] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.202716 17621 raft_consensus.cc:504] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.204816 17713 leader_election.cc:288] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.204957 17427 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.206861 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "946149c123e84f048c41fe4f86395165" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.209874 17484 consensus_queue.cc:1035] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:04:13.210505 17277 leader_election.cc:341] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.204886 17621 leader_election.cc:288] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.209869 17427 ts_tablet_manager.cc:1367] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.106s user 0.015s sys 0.012s
I20240501 08:04:13.208428 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa513fcceeb4a15b0a193ddf00a6e62" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:13.212742 17231 raft_consensus.cc:3037] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.214488 17427 raft_consensus.cc:348] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.215305 17427 raft_consensus.cc:374] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.215603 17427 raft_consensus.cc:729] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.216292 17427 consensus_queue.cc:260] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.218362 17427 ts_tablet_manager.cc:1398] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:13.219316 17427 tablet_bootstrap.cc:492] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.220145 17231 raft_consensus.cc:2454] T 4aa513fcceeb4a15b0a193ddf00a6e62 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:13.219617 17663 raft_consensus.cc:482] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.205367 17278 leader_election.cc:302] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1; no voters: fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.205201 17428 ts_tablet_manager.cc:1367] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.118s user 0.010s sys 0.022s
I20240501 08:04:13.222282 17432 raft_consensus.cc:2781] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.222761 17432 raft_consensus.cc:686] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:13.223551 17432 consensus_queue.cc:237] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.225569 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "946149c123e84f048c41fe4f86395165" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.223058 17663 raft_consensus.cc:504] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.226655 17428 raft_consensus.cc:348] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.228777 17428 raft_consensus.cc:374] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.230197 17428 raft_consensus.cc:729] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.230422 17427 tablet_bootstrap.cc:654] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:13.231900 17278 leader_election.cc:341] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.232368 17278 leader_election.cc:302] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.233745 17351 leader_election.cc:302] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.233481 17712 raft_consensus.cc:482] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.231350 17428 consensus_queue.cc:260] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.234920 17712 raft_consensus.cc:504] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.235185 17714 raft_consensus.cc:2726] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.237291 17704 raft_consensus.cc:2781] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.246516 17426 tablet_bootstrap.cc:492] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.247069 17426 ts_tablet_manager.cc:1367] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.106s user 0.007s sys 0.003s
I20240501 08:04:13.237052 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d9ba955a5347ac9cc55e89b2b42e4e" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.237751 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa513fcceeb4a15b0a193ddf00a6e62" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.251219 17428 ts_tablet_manager.cc:1398] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.029s user 0.005s sys 0.002s
I20240501 08:04:13.252210 17428 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:13.253618 17712 leader_election.cc:288] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.254596 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d9ba955a5347ac9cc55e89b2b42e4e" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.254729 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06988f9531b4744b9859548c8cc6530" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.244386 17663 leader_election.cc:288] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:13.259096 17203 leader_election.cc:341] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.256100 17307 raft_consensus.cc:3037] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:13.260322 17202 leader_election.cc:341] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.261080 17202 leader_election.cc:302] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.255389 17230 raft_consensus.cc:2454] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:13.263741 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06988f9531b4744b9859548c8cc6530" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.263943 17351 leader_election.cc:302] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.255874 17704 raft_consensus.cc:686] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:13.264250 17619 raft_consensus.cc:2726] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.266086 17663 consensus_queue.cc:1035] T 0274d08768ff4a1892e81b59d1677f2a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20240501 08:04:13.267428 17503 raft_consensus.cc:2781] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.275714 17503 raft_consensus.cc:482] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.276043 17503 raft_consensus.cc:3037] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.280475 17704 consensus_queue.cc:237] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:13.282035 17351 leader_election.cc:341] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.296049 17432 raft_consensus.cc:482] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.296532 17432 raft_consensus.cc:504] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.298322 17432 leader_election.cc:288] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.306458 17132 catalog_manager.cc:5484] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:13.308105 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fe90f0956c48d184cd48f76c7e2a64" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.309356 17385 raft_consensus.cc:3037] T 4437b60cbdcd4b7d850f3534a8845e92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.311151 17428 tablet_bootstrap.cc:654] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.301357 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fe90f0956c48d184cd48f76c7e2a64" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
W20240501 08:04:13.314079 17278 leader_election.cc:341] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:04:13.326221 17277 leader_election.cc:341] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.326750 17277 leader_election.cc:302] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.330879 17307 raft_consensus.cc:2454] T 4aa513fcceeb4a15b0a193ddf00a6e62 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:13.331804 17432 raft_consensus.cc:2726] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.335178 17427 tablet_bootstrap.cc:492] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.335620 17427 ts_tablet_manager.cc:1367] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.116s user 0.011s sys 0.002s
I20240501 08:04:13.337918 17427 raft_consensus.cc:348] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.338579 17427 raft_consensus.cc:374] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.338836 17427 raft_consensus.cc:729] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.339390 17427 consensus_queue.cc:260] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.339655 17757 consensus_queue.cc:1035] T 4437b60cbdcd4b7d850f3534a8845e92 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:13.339772 17426 raft_consensus.cc:348] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.340994 17426 raft_consensus.cc:374] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.341514 17426 raft_consensus.cc:729] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.342895 17426 consensus_queue.cc:260] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.345984 17427 ts_tablet_manager.cc:1398] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20240501 08:04:13.348562 17427 tablet_bootstrap.cc:492] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.338037 17134 catalog_manager.cc:5484] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:13.348585 17426 ts_tablet_manager.cc:1398] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.100s user 0.004s sys 0.002s
I20240501 08:04:13.350757 17426 tablet_bootstrap.cc:492] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.355183 17503 raft_consensus.cc:504] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.357062 17503 leader_election.cc:288] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.358229 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06988f9531b4744b9859548c8cc6530" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:13.359903 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06988f9531b4744b9859548c8cc6530" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.360145 17231 raft_consensus.cc:3037] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:13.361693 17351 leader_election.cc:341] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.370595 17231 raft_consensus.cc:2454] T e06988f9531b4744b9859548c8cc6530 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:13.374419 17351 leader_election.cc:302] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.377317 17704 raft_consensus.cc:2781] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.377887 17704 raft_consensus.cc:686] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:13.378053 17428 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:13.378700 17428 ts_tablet_manager.cc:1367] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.127s user 0.011s sys 0.006s
I20240501 08:04:13.378695 17704 consensus_queue.cc:237] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.382536 17433 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:13.385694 17428 raft_consensus.cc:348] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.386366 17428 raft_consensus.cc:374] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.386623 17428 raft_consensus.cc:729] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.387221 17428 consensus_queue.cc:260] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.389333 17428 ts_tablet_manager.cc:1398] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20240501 08:04:13.390388 17428 tablet_bootstrap.cc:492] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:13.392851 17426 tablet_bootstrap.cc:654] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.395789 17759 raft_consensus.cc:482] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.396173 17427 tablet_bootstrap.cc:654] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.396270 17759 raft_consensus.cc:504] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.399188 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
W20240501 08:04:13.400280 17277 leader_election.cc:341] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.402963 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
W20240501 08:04:13.405125 17278 leader_election.cc:341] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.405543 17278 leader_election.cc:302] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.405303 17135 catalog_manager.cc:5484] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:13.413599 17621 raft_consensus.cc:2726] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.415679 17503 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:13.422883 17663 raft_consensus.cc:482] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.423336 17663 raft_consensus.cc:504] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.425129 17663 leader_election.cc:288] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.427059 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.401058 17759 leader_election.cc:288] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:13.429045 17203 leader_election.cc:341] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.430382 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:13.432705 17202 leader_election.cc:341] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.433140 17202 leader_election.cc:302] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.434726 17427 tablet_bootstrap.cc:492] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.435154 17427 ts_tablet_manager.cc:1367] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.018s
I20240501 08:04:13.438130 17663 raft_consensus.cc:2726] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.446936 17427 raft_consensus.cc:348] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.447662 17427 raft_consensus.cc:374] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.447933 17427 raft_consensus.cc:729] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.448539 17427 consensus_queue.cc:260] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.453115 17428 tablet_bootstrap.cc:654] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.454232 17667 raft_consensus.cc:482] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.454774 17427 ts_tablet_manager.cc:1398] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20240501 08:04:13.455682 17667 raft_consensus.cc:504] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.456462 17427 tablet_bootstrap.cc:492] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.457720 17667 leader_election.cc:288] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.460302 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3869c361a6d47679025e9c5e47b4706" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.460793 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3869c361a6d47679025e9c5e47b4706" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.460978 17384 raft_consensus.cc:2454] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
W20240501 08:04:13.462263 17202 leader_election.cc:341] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.463127 17427 tablet_bootstrap.cc:654] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.463392 17203 leader_election.cc:302] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.464169 17667 raft_consensus.cc:2781] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.464514 17667 raft_consensus.cc:482] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.464836 17667 raft_consensus.cc:3037] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.471266 17426 tablet_bootstrap.cc:492] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.477790 17426 ts_tablet_manager.cc:1367] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.128s user 0.022s sys 0.013s
I20240501 08:04:13.480087 17426 raft_consensus.cc:348] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.480755 17426 raft_consensus.cc:374] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.481016 17426 raft_consensus.cc:729] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.481163 17667 raft_consensus.cc:504] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:13.483812 17203 consensus_peers.cc:485] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e -> Peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Couldn't send request to peer 5655c2e9168d4eeaa212db8874c2efa5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:13.484208 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3869c361a6d47679025e9c5e47b4706" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.482400 17426 consensus_queue.cc:260] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.485777 17426 ts_tablet_manager.cc:1398] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
W20240501 08:04:13.485719 17202 leader_election.cc:341] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.486707 17426 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.498872 17426 tablet_bootstrap.cc:654] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.498907 17484 consensus_queue.cc:1035] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:13.500244 17667 leader_election.cc:288] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.500416 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3869c361a6d47679025e9c5e47b4706" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:13.500984 17384 raft_consensus.cc:3037] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.502653 17428 tablet_bootstrap.cc:492] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:13.502768 17427 tablet_bootstrap.cc:492] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.503298 17427 ts_tablet_manager.cc:1367] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.047s user 0.019s sys 0.005s
I20240501 08:04:13.505895 17428 ts_tablet_manager.cc:1367] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.116s user 0.010s sys 0.000s
I20240501 08:04:13.505823 17427 raft_consensus.cc:348] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.506613 17427 raft_consensus.cc:374] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.506899 17427 raft_consensus.cc:729] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.507483 17427 consensus_queue.cc:260] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.509938 17427 ts_tablet_manager.cc:1398] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:04:13.509847 17428 raft_consensus.cc:348] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.510685 17428 raft_consensus.cc:374] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.510800 17427 tablet_bootstrap.cc:492] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.509640 17384 raft_consensus.cc:2454] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:13.512166 17428 raft_consensus.cc:729] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.512926 17428 consensus_queue.cc:260] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.515013 17428 ts_tablet_manager.cc:1398] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20240501 08:04:13.515359 17203 leader_election.cc:302] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.515903 17427 tablet_bootstrap.cc:654] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.520164 17484 raft_consensus.cc:2781] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.521373 17484 raft_consensus.cc:686] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:13.522205 17484 consensus_queue.cc:237] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.518399 17426 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.524432 17426 ts_tablet_manager.cc:1367] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.004s
I20240501 08:04:13.525460 17759 raft_consensus.cc:482] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.525936 17759 raft_consensus.cc:504] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.527688 17759 leader_election.cc:288] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.528956 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d9ba955a5347ac9cc55e89b2b42e4e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.529577 17230 raft_consensus.cc:2454] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:13.529798 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d9ba955a5347ac9cc55e89b2b42e4e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.530666 17277 leader_election.cc:302] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.535962 17759 raft_consensus.cc:2781] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.536324 17759 raft_consensus.cc:482] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.536641 17759 raft_consensus.cc:3037] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.530436 17384 raft_consensus.cc:2454] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:13.546741 17759 raft_consensus.cc:504] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.532523 17426 raft_consensus.cc:348] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.548314 17426 raft_consensus.cc:374] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.548770 17426 raft_consensus.cc:729] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.548663 17713 consensus_queue.cc:1035] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:13.549507 17426 consensus_queue.cc:260] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.558210 17621 raft_consensus.cc:482] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.558737 17621 raft_consensus.cc:504] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.560703 17621 leader_election.cc:288] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.562106 17231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.562431 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.562796 17231 raft_consensus.cc:2454] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:13.557796 17426 ts_tablet_manager.cc:1398] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20240501 08:04:13.564082 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d9ba955a5347ac9cc55e89b2b42e4e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
W20240501 08:04:13.564359 17278 leader_election.cc:341] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.564808 17232 raft_consensus.cc:3037] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.564993 17428 tablet_bootstrap.cc:492] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:13.565560 17277 leader_election.cc:302] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:13.566673 17621 raft_consensus.cc:2781] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.567035 17621 raft_consensus.cc:482] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.567348 17621 raft_consensus.cc:3037] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.569002 17759 leader_election.cc:288] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.564324 17426 tablet_bootstrap.cc:492] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.572786 17232 raft_consensus.cc:2454] T 35d9ba955a5347ac9cc55e89b2b42e4e P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:13.576989 17663 raft_consensus.cc:482] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.578526 17277 leader_election.cc:302] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.579412 17667 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.580228 17759 raft_consensus.cc:2781] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.581959 17667 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.582054 17621 raft_consensus.cc:504] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.577524 17663 raft_consensus.cc:504] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.582365 17759 raft_consensus.cc:686] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:13.580787 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d9ba955a5347ac9cc55e89b2b42e4e" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:13.585608 17384 raft_consensus.cc:3037] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.589860 17663 leader_election.cc:288] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.590548 17667 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.590853 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.591046 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.591490 17308 raft_consensus.cc:2454] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:13.592084 17759 consensus_queue.cc:237] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:13.592895 17203 leader_election.cc:341] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.593314 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.593858 17386 raft_consensus.cc:2454] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:13.593888 17202 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.595137 17667 raft_consensus.cc:2781] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240501 08:04:13.595474 17667 raft_consensus.cc:482] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.595779 17667 raft_consensus.cc:3037] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:04:13.597229 17384 raft_consensus.cc:2454] T 35d9ba955a5347ac9cc55e89b2b42e4e P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:13.598680 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.600600 17428 tablet_bootstrap.cc:654] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:13.602843 17202 leader_election.cc:341] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.603044 17427 tablet_bootstrap.cc:492] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.603257 17202 leader_election.cc:302] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:13.603461 17427 ts_tablet_manager.cc:1367] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.093s user 0.018s sys 0.042s
I20240501 08:04:13.604661 17663 raft_consensus.cc:2726] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.605677 17427 raft_consensus.cc:348] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.606338 17427 raft_consensus.cc:374] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.606578 17427 raft_consensus.cc:729] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.607159 17427 consensus_queue.cc:260] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.608361 17426 tablet_bootstrap.cc:654] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.609093 17427 ts_tablet_manager.cc:1398] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240501 08:04:13.609831 17427 tablet_bootstrap.cc:492] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.611723 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:13.612356 17232 raft_consensus.cc:3037] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.612728 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:13.615401 17135 catalog_manager.cc:5484] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:13.613901 17667 raft_consensus.cc:504] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:13.619714 17278 leader_election.cc:341] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:04:13.620254 17351 consensus_peers.cc:485] T 867eb9d6d70b41ddb63cd0e79921d391 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Couldn't send request to peer cdccadabd9154ea69e15405d75301fc1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:13.621491 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.622049 17307 raft_consensus.cc:3037] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:04:13.602872 17621 leader_election.cc:288] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.640290 17713 raft_consensus.cc:482] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.641258 17427 tablet_bootstrap.cc:654] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.647135 17132 catalog_manager.cc:5484] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:13.648103 17713 raft_consensus.cc:504] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.652760 17713 leader_election.cc:288] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.653681 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "946149c123e84f048c41fe4f86395165" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.655275 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "946149c123e84f048c41fe4f86395165" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.655886 17230 raft_consensus.cc:2454] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:13.656095 17307 raft_consensus.cc:2454] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 2.
I20240501 08:04:13.657841 17351 leader_election.cc:302] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.654310 17308 raft_consensus.cc:2454] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 0.
I20240501 08:04:13.661548 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d9db161d31444f9a07bf586dff9a45a" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:13.657546 17667 leader_election.cc:288] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 2 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.663713 17202 leader_election.cc:302] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.663107 17384 raft_consensus.cc:3037] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:04:13.675108 17427 tablet_bootstrap.cc:492] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.675669 17713 consensus_queue.cc:1035] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:13.675652 17427 ts_tablet_manager.cc:1367] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.066s user 0.022s sys 0.001s
I20240501 08:04:13.657586 17232 raft_consensus.cc:2454] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:13.667865 17426 tablet_bootstrap.cc:492] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.677681 17484 raft_consensus.cc:2781] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:04:13.678138 17484 raft_consensus.cc:686] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [term 2 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:13.678280 17277 leader_election.cc:302] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:13.677700 17426 ts_tablet_manager.cc:1367] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.114s user 0.017s sys 0.016s
I20240501 08:04:13.659221 17712 raft_consensus.cc:2781] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.681036 17427 raft_consensus.cc:348] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.680868 17712 raft_consensus.cc:482] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.681749 17427 raft_consensus.cc:374] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.682070 17712 raft_consensus.cc:3037] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.682066 17427 raft_consensus.cc:729] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.681953 17426 raft_consensus.cc:348] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.683056 17427 consensus_queue.cc:260] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.684370 17484 consensus_queue.cc:237] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.685063 17427 ts_tablet_manager.cc:1398] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20240501 08:04:13.685966 17427 tablet_bootstrap.cc:492] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.686026 17621 raft_consensus.cc:2781] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.686534 17621 raft_consensus.cc:686] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:13.687278 17621 consensus_queue.cc:237] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.690574 17426 raft_consensus.cc:374] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.690872 17426 raft_consensus.cc:729] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.691215 17466 raft_consensus.cc:482] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.691665 17426 consensus_queue.cc:260] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.691679 17466 raft_consensus.cc:504] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.692270 17427 tablet_bootstrap.cc:654] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.694763 17466 leader_election.cc:288] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.695896 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.696156 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:13.696395 17426 ts_tablet_manager.cc:1398] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20240501 08:04:13.696739 17307 raft_consensus.cc:2454] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
W20240501 08:04:13.697916 17203 leader_election.cc:341] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.676507 17384 raft_consensus.cc:2454] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 2.
I20240501 08:04:13.698678 17202 leader_election.cc:302] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:13.699951 17426 tablet_bootstrap.cc:492] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.700210 17712 raft_consensus.cc:504] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.702040 17428 tablet_bootstrap.cc:492] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:13.702504 17428 ts_tablet_manager.cc:1367] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.138s user 0.016s sys 0.003s
I20240501 08:04:13.707121 17484 raft_consensus.cc:2781] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.710978 17232 raft_consensus.cc:3037] T a8945347fd484e038a72f8580fc88472 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.707538 17484 raft_consensus.cc:482] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.717744 17484 raft_consensus.cc:3037] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.722748 17426 tablet_bootstrap.cc:654] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.724411 17759 consensus_queue.cc:1035] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:13.727969 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "946149c123e84f048c41fe4f86395165" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:13.728575 17237 raft_consensus.cc:3037] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.736244 17237 raft_consensus.cc:2454] T 946149c123e84f048c41fe4f86395165 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:13.737125 17426 tablet_bootstrap.cc:492] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.737320 17351 leader_election.cc:302] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.737640 17426 ts_tablet_manager.cc:1367] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.038s user 0.013s sys 0.004s
I20240501 08:04:13.738297 17713 raft_consensus.cc:2781] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.726271 17428 raft_consensus.cc:348] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.739955 17428 raft_consensus.cc:374] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.740272 17428 raft_consensus.cc:729] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.739384 17713 raft_consensus.cc:686] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Becoming Leader. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Running, Role: LEADER
I20240501 08:04:13.742010 17713 consensus_queue.cc:237] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.743578 17426 raft_consensus.cc:348] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.744277 17426 raft_consensus.cc:374] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.744518 17426 raft_consensus.cc:729] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.745132 17426 consensus_queue.cc:260] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.739822 17134 catalog_manager.cc:5484] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 2, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 2 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:13.753505 17428 consensus_queue.cc:260] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.780563 17426 ts_tablet_manager.cc:1398] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20240501 08:04:13.789870 17135 catalog_manager.cc:5484] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:13.792481 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "946149c123e84f048c41fe4f86395165" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.793098 17308 raft_consensus.cc:3037] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.800029 17427 tablet_bootstrap.cc:492] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.785614 17621 raft_consensus.cc:482] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.801218 17621 raft_consensus.cc:504] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.805050 17621 leader_election.cc:288] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.807904 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3869c361a6d47679025e9c5e47b4706" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.801191 17427 ts_tablet_manager.cc:1367] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.115s user 0.020s sys 0.017s
I20240501 08:04:13.808213 17308 raft_consensus.cc:2454] T 946149c123e84f048c41fe4f86395165 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5655c2e9168d4eeaa212db8874c2efa5 in term 1.
I20240501 08:04:13.818544 17428 ts_tablet_manager.cc:1398] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.116s user 0.006s sys 0.012s
I20240501 08:04:13.822861 17428 tablet_bootstrap.cc:492] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:13.824406 17427 raft_consensus.cc:348] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.825574 17427 raft_consensus.cc:374] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.825942 17427 raft_consensus.cc:729] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.826849 17427 consensus_queue.cc:260] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.829792 17312 raft_consensus.cc:3037] T e06988f9531b4744b9859548c8cc6530 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.832502 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3869c361a6d47679025e9c5e47b4706" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:13.834244 17426 tablet_bootstrap.cc:492] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:13.834597 17386 raft_consensus.cc:2379] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:13.786691 17712 leader_election.cc:288] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:13.855993 17427 ts_tablet_manager.cc:1398] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.042s user 0.009s sys 0.000s
I20240501 08:04:13.860460 17427 tablet_bootstrap.cc:492] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:13.832571 17755 raft_consensus.cc:482] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:13.866755 17278 leader_election.cc:302] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:13.835218 17432 consensus_queue.cc:1035] T a8945347fd484e038a72f8580fc88472 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20240501 08:04:13.871212 17621 raft_consensus.cc:3037] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.870087 17755 raft_consensus.cc:504] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.896317 17426 tablet_bootstrap.cc:654] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.897356 17755 leader_election.cc:288] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.902578 17428 tablet_bootstrap.cc:654] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.893086 17132 catalog_manager.cc:5484] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131). New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } }
I20240501 08:04:13.904875 17484 raft_consensus.cc:504] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.908126 17484 leader_election.cc:288] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.912454 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a64317c43cd4a2598c392879c3ff65a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
W20240501 08:04:13.922088 17278 leader_election.cc:341] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.923883 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:13.924461 17312 raft_consensus.cc:3037] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.924782 17713 consensus_queue.cc:1035] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20240501 08:04:13.927832 17237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a64317c43cd4a2598c392879c3ff65a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:13.928457 17237 raft_consensus.cc:2454] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:13.929373 17503 consensus_queue.cc:1035] T e06988f9531b4744b9859548c8cc6530 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:13.938086 17277 leader_election.cc:302] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:13.938757 17312 raft_consensus.cc:2454] T 96c8eff63a7449018719d17f779939c5 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:13.940542 17755 raft_consensus.cc:2781] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:13.940944 17755 raft_consensus.cc:482] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:13.941283 17755 raft_consensus.cc:3037] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:13.929004 17713 consensus_queue.cc:1035] T 4aa513fcceeb4a15b0a193ddf00a6e62 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20240501 08:04:13.934271 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:13.949295 17427 tablet_bootstrap.cc:654] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:13.942561 17202 leader_election.cc:302] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:13.951057 17663 raft_consensus.cc:2781] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:13.955525 17663 raft_consensus.cc:686] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:13.968374 17663 consensus_queue.cc:237] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.969199 17621 raft_consensus.cc:2726] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:13.976043 17426 tablet_bootstrap.cc:492] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:13.976509 17426 ts_tablet_manager.cc:1367] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.143s user 0.018s sys 0.014s
I20240501 08:04:13.975018 17755 raft_consensus.cc:504] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.978847 17427 tablet_bootstrap.cc:492] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:13.980594 17427 ts_tablet_manager.cc:1367] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.120s user 0.019s sys 0.013s
I20240501 08:04:13.982017 17755 leader_election.cc:288] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:13.982882 17427 raft_consensus.cc:348] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.983568 17427 raft_consensus.cc:374] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.983844 17427 raft_consensus.cc:729] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.984412 17427 consensus_queue.cc:260] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.985847 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a64317c43cd4a2598c392879c3ff65a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
W20240501 08:04:13.987221 17278 leader_election.cc:341] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:13.978855 17426 raft_consensus.cc:348] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.988816 17426 raft_consensus.cc:374] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:13.989082 17426 raft_consensus.cc:729] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:13.989746 17426 consensus_queue.cc:260] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:13.991869 17426 ts_tablet_manager.cc:1398] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20240501 08:04:13.992363 17427 ts_tablet_manager.cc:1398] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20240501 08:04:13.993297 17427 tablet_bootstrap.cc:492] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:14.003698 17134 catalog_manager.cc:5484] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:14.008641 17426 tablet_bootstrap.cc:492] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:14.015842 17427 tablet_bootstrap.cc:654] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.020479 17621 consensus_queue.cc:1035] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:14.024925 17427 tablet_bootstrap.cc:492] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:14.026993 17426 tablet_bootstrap.cc:654] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.028395 17306 raft_consensus.cc:3037] T 867eb9d6d70b41ddb63cd0e79921d391 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.029500 17427 ts_tablet_manager.cc:1367] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.001s
I20240501 08:04:14.031788 17427 raft_consensus.cc:348] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.032434 17427 raft_consensus.cc:374] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.032711 17427 raft_consensus.cc:729] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.033903 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a64317c43cd4a2598c392879c3ff65a" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:14.034554 17230 raft_consensus.cc:3037] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.037545 17427 consensus_queue.cc:260] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.040074 17427 ts_tablet_manager.cc:1398] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20240501 08:04:14.041257 17230 raft_consensus.cc:2454] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:14.045789 17466 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:04:14.055148 17203 leader_election.cc:341] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.057300 17428 tablet_bootstrap.cc:492] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:14.058187 17428 ts_tablet_manager.cc:1367] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.236s user 0.016s sys 0.024s
I20240501 08:04:14.060452 17428 raft_consensus.cc:348] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.061091 17428 raft_consensus.cc:374] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.061363 17428 raft_consensus.cc:729] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.061988 17428 consensus_queue.cc:260] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.062999 17427 tablet_bootstrap.cc:492] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:14.068365 17427 tablet_bootstrap.cc:654] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.073640 17428 ts_tablet_manager.cc:1398] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20240501 08:04:14.074548 17428 tablet_bootstrap.cc:492] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:14.079490 17428 tablet_bootstrap.cc:654] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.088938 17426 tablet_bootstrap.cc:492] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:14.089408 17426 ts_tablet_manager.cc:1367] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.081s user 0.010s sys 0.007s
I20240501 08:04:14.091661 17426 raft_consensus.cc:348] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.092314 17426 raft_consensus.cc:374] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.092581 17426 raft_consensus.cc:729] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.095866 17277 leader_election.cc:302] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:14.096745 17760 raft_consensus.cc:2781] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:14.097268 17760 raft_consensus.cc:686] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:14.097579 17426 consensus_queue.cc:260] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.098038 17760 consensus_queue.cc:237] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.102175 17426 ts_tablet_manager.cc:1398] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.012s user 0.004s sys 0.002s
I20240501 08:04:14.112694 17426 tablet_bootstrap.cc:492] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:14.118217 17426 tablet_bootstrap.cc:654] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.124353 17621 consensus_queue.cc:1035] T 35d9ba955a5347ac9cc55e89b2b42e4e P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:14.124738 17426 tablet_bootstrap.cc:492] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:14.125200 17426 ts_tablet_manager.cc:1367] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.001s
I20240501 08:04:14.127728 17428 tablet_bootstrap.cc:492] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:14.128198 17428 ts_tablet_manager.cc:1367] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.054s user 0.019s sys 0.010s
I20240501 08:04:14.129837 17426 raft_consensus.cc:348] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.130569 17426 raft_consensus.cc:374] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.130816 17426 raft_consensus.cc:729] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.131877 17426 consensus_queue.cc:260] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.132019 17428 raft_consensus.cc:348] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.132685 17428 raft_consensus.cc:374] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.133051 17428 raft_consensus.cc:729] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.134464 17426 ts_tablet_manager.cc:1398] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.009s user 0.001s sys 0.004s
I20240501 08:04:14.134179 17484 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:14.135524 17426 tablet_bootstrap.cc:492] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:14.132632 17466 raft_consensus.cc:482] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.135975 17466 raft_consensus.cc:504] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.136408 17428 consensus_queue.cc:260] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.140069 17466 leader_election.cc:288] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.140466 17426 tablet_bootstrap.cc:654] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.140719 17428 ts_tablet_manager.cc:1398] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240501 08:04:14.148113 17428 tablet_bootstrap.cc:492] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:14.153334 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.155011 17428 tablet_bootstrap.cc:654] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.158140 17713 raft_consensus.cc:482] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.158720 17713 raft_consensus.cc:504] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.159945 17427 tablet_bootstrap.cc:492] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:14.160375 17427 ts_tablet_manager.cc:1367] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.098s user 0.005s sys 0.036s
W20240501 08:04:14.161952 17202 leader_election.cc:341] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.162683 17713 leader_election.cc:288] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:14.162660 17427 raft_consensus.cc:348] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.161682 17788 consensus_queue.cc:1035] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:14.164395 17427 raft_consensus.cc:374] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.164670 17427 raft_consensus.cc:729] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.165200 17427 consensus_queue.cc:260] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.167156 17427 ts_tablet_manager.cc:1398] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 08:04:14.168170 17427 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: Bootstrap starting.
I20240501 08:04:14.169133 17312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.167624 17134 catalog_manager.cc:5484] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:14.173863 17427 tablet_bootstrap.cc:654] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.217372 17306 raft_consensus.cc:1261] T 0d9db161d31444f9a07bf586dff9a45a P cdccadabd9154ea69e15405d75301fc1 [term 2 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240501 08:04:14.227160 17619 consensus_queue.cc:1035] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:14.231180 17807 raft_consensus.cc:482] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.232896 17807 raft_consensus.cc:504] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:14.241822 17203 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.131:43915, user_credentials={real_user=slave}} blocked reactor thread for 112012us
I20240501 08:04:14.269778 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9343b763cf7743e88400ddcd42a6b931" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:14.270903 17233 raft_consensus.cc:2379] T 9343b763cf7743e88400ddcd42a6b931 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:14.272393 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.273077 17381 raft_consensus.cc:2454] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:14.282430 17621 consensus_queue.cc:1035] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:04:14.283221 17183 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240501 08:04:14.290683 17621 raft_consensus.cc:482] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.291229 17621 raft_consensus.cc:504] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.293504 17427 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: No bootstrap required, opened a new log
I20240501 08:04:14.293967 17427 ts_tablet_manager.cc:1367] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: Time spent bootstrapping tablet: real 0.126s user 0.029s sys 0.034s
I20240501 08:04:14.293545 17621 leader_election.cc:288] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.296371 17427 raft_consensus.cc:348] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.297013 17427 raft_consensus.cc:374] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.297268 17427 raft_consensus.cc:729] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.298229 17427 consensus_queue.cc:260] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.300403 17427 ts_tablet_manager.cc:1398] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:14.303751 17351 leader_election.cc:302] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:14.305191 17426 tablet_bootstrap.cc:492] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:14.305480 17428 tablet_bootstrap.cc:492] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:14.317497 17426 ts_tablet_manager.cc:1367] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.182s user 0.052s sys 0.094s
I20240501 08:04:14.317904 17428 ts_tablet_manager.cc:1367] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.170s user 0.027s sys 0.057s
I20240501 08:04:14.305814 17788 raft_consensus.cc:3037] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.319826 17426 raft_consensus.cc:348] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.320375 17428 raft_consensus.cc:348] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.321138 17428 raft_consensus.cc:374] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.321414 17428 raft_consensus.cc:729] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.322248 17428 consensus_queue.cc:260] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.320523 17426 raft_consensus.cc:374] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.324411 17426 raft_consensus.cc:729] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.325078 17428 ts_tablet_manager.cc:1398] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20240501 08:04:14.325071 17426 consensus_queue.cc:260] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.327302 17426 ts_tablet_manager.cc:1398] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.009s user 0.003s sys 0.002s
I20240501 08:04:14.328218 17426 tablet_bootstrap.cc:492] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:14.310941 17381 raft_consensus.cc:1261] T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5 [term 2 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240501 08:04:14.333850 17428 tablet_bootstrap.cc:492] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:14.337076 17203 leader_election.cc:302] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:14.337777 17484 raft_consensus.cc:2781] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:14.338093 17484 raft_consensus.cc:482] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:14.338392 17484 raft_consensus.cc:3037] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.340981 17807 leader_election.cc:288] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.342036 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.345016 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3abec6d380a4a5fa72fd34fe37bbd92" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.345947 17484 raft_consensus.cc:504] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.345719 17307 raft_consensus.cc:2454] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:14.349419 17202 leader_election.cc:302] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:14.349880 17484 leader_election.cc:288] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.356750 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:14.367527 17426 tablet_bootstrap.cc:654] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.367172 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:14.372066 17484 raft_consensus.cc:2781] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:14.372414 17484 raft_consensus.cc:482] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:14.372711 17484 raft_consensus.cc:3037] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:14.369598 17278 leader_election.cc:341] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:04:14.373485 17202 leader_election.cc:341] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.383419 17788 raft_consensus.cc:2726] T 9343b763cf7743e88400ddcd42a6b931 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:14.367244 17663 consensus_queue.cc:1035] T 0d9db161d31444f9a07bf586dff9a45a P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20240501 08:04:14.377275 17667 raft_consensus.cc:482] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.379528 17619 raft_consensus.cc:482] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.387629 17667 raft_consensus.cc:504] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.387782 17619 raft_consensus.cc:504] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.388193 17663 raft_consensus.cc:482] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.388633 17663 raft_consensus.cc:504] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.389503 17667 leader_election.cc:288] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.389681 17619 leader_election.cc:288] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.391783 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
W20240501 08:04:14.393832 17202 leader_election.cc:341] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.395879 17484 raft_consensus.cc:504] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.399027 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fe90f0956c48d184cd48f76c7e2a64" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.375857 17230 raft_consensus.cc:2454] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:14.402860 17484 leader_election.cc:288] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.408388 17386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3abec6d380a4a5fa72fd34fe37bbd92" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
W20240501 08:04:14.411298 17203 leader_election.cc:341] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.413008 17277 leader_election.cc:302] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:14.377768 17791 consensus_queue.cc:1035] T 946149c123e84f048c41fe4f86395165 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20240501 08:04:14.400599 17385 raft_consensus.cc:2454] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:14.413766 17760 raft_consensus.cc:2781] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:14.414137 17760 raft_consensus.cc:482] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:14.414070 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fe90f0956c48d184cd48f76c7e2a64" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.414608 17760 raft_consensus.cc:3037] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.408200 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.416313 17203 leader_election.cc:302] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:14.416831 17466 raft_consensus.cc:2781] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20240501 08:04:14.416893 17203 leader_election.cc:341] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.417099 17466 raft_consensus.cc:482] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:14.417371 17203 leader_election.cc:302] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5, cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:14.417467 17466 raft_consensus.cc:3037] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.417876 17484 raft_consensus.cc:2726] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:14.415124 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3abec6d380a4a5fa72fd34fe37bbd92" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:14.418521 17310 raft_consensus.cc:3037] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.423161 17379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:14.423748 17379 raft_consensus.cc:3037] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.425640 17466 raft_consensus.cc:504] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.430694 17466 leader_election.cc:288] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.406596 17759 consensus_queue.cc:1035] T 9343b763cf7743e88400ddcd42a6b931 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20240501 08:04:14.452435 17426 tablet_bootstrap.cc:492] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:14.433005 17304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fe90f0956c48d184cd48f76c7e2a64" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:14.452925 17426 ts_tablet_manager.cc:1367] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.125s user 0.020s sys 0.008s
I20240501 08:04:14.424875 17305 raft_consensus.cc:2454] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
W20240501 08:04:14.493875 17203 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.131:43915, user_credentials={real_user=slave}} blocked reactor thread for 70731.2us
I20240501 08:04:14.531227 17426 raft_consensus.cc:348] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.532011 17426 raft_consensus.cc:374] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.532209 17426 raft_consensus.cc:729] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.532629 17426 consensus_queue.cc:260] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.538074 17426 ts_tablet_manager.cc:1398] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.085s user 0.020s sys 0.010s
I20240501 08:04:14.538869 17426 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: Bootstrap starting.
I20240501 08:04:14.523460 17380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3abec6d380a4a5fa72fd34fe37bbd92" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.523891 17713 raft_consensus.cc:482] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.550784 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fe90f0956c48d184cd48f76c7e2a64" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:14.551395 17384 raft_consensus.cc:3037] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.533633 17788 raft_consensus.cc:482] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.553555 17788 raft_consensus.cc:504] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.558318 17788 leader_election.cc:288] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:14.572106 17309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee19995987e4749b8c949a60e1ae1e9" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.572649 17309 raft_consensus.cc:2454] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 0.
I20240501 08:04:14.573439 17202 leader_election.cc:302] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:14.575135 17807 raft_consensus.cc:2781] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:14.575501 17807 raft_consensus.cc:482] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:14.575803 17807 raft_consensus.cc:3037] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.580107 17310 raft_consensus.cc:2454] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:14.567960 17663 leader_election.cc:288] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.585287 17760 raft_consensus.cc:504] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.586344 17428 tablet_bootstrap.cc:654] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.543299 17713 raft_consensus.cc:504] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
W20240501 08:04:14.588310 17351 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.130:33757, user_credentials={real_user=slave}} blocked reactor thread for 53011.4us
W20240501 08:04:14.591441 17203 leader_election.cc:341] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.594854 17202 leader_election.cc:302] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
I20240501 08:04:14.595887 17484 raft_consensus.cc:2781] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:14.596408 17484 raft_consensus.cc:686] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:14.598320 17484 consensus_queue.cc:237] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.600416 17713 leader_election.cc:288] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:14.601080 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a64317c43cd4a2598c392879c3ff65a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.603693 17383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee19995987e4749b8c949a60e1ae1e9" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.604781 17760 leader_election.cc:288] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.607178 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:14.607210 17383 raft_consensus.cc:3037] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.607766 17232 raft_consensus.cc:3037] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.610612 17384 raft_consensus.cc:2454] T a0fe90f0956c48d184cd48f76c7e2a64 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:14.613631 17232 raft_consensus.cc:2454] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:14.617374 17466 consensus_queue.cc:1035] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:14.618525 17807 raft_consensus.cc:504] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.620043 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a64317c43cd4a2598c392879c3ff65a" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:14.620774 17230 raft_consensus.cc:2379] T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:14.621541 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:14.626518 17426 tablet_bootstrap.cc:654] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.626533 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12aee8e217041f19aa8102e5fe1507f" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:14.627843 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:14.634778 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee19995987e4749b8c949a60e1ae1e9" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1"
I20240501 08:04:14.635421 17310 raft_consensus.cc:3037] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:04:14.636219 17278 leader_election.cc:341] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.640272 17807 leader_election.cc:288] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Requested vote from peers cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:14.641029 17379 raft_consensus.cc:2454] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:14.633524 17760 consensus_queue.cc:1035] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:04:14.633863 17819 log.cc:924] Time spent T 0d9db161d31444f9a07bf586dff9a45a P 5655c2e9168d4eeaa212db8874c2efa5: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20240501 08:04:14.645316 17310 raft_consensus.cc:2454] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa9d3bc74a68481e9de647223cb0110e in term 1.
I20240501 08:04:14.646471 17202 leader_election.cc:302] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:14.647435 17807 raft_consensus.cc:2781] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:14.648214 17277 leader_election.cc:302] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters: 5655c2e9168d4eeaa212db8874c2efa5
W20240501 08:04:14.650566 17203 consensus_peers.cc:485] T 96c8eff63a7449018719d17f779939c5 P fa9d3bc74a68481e9de647223cb0110e -> Peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Couldn't send request to peer 5655c2e9168d4eeaa212db8874c2efa5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20240501 08:04:14.652065 17203 leader_election.cc:341] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.652912 17203 leader_election.cc:302] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:14.654450 17760 raft_consensus.cc:2781] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:14.655239 17466 raft_consensus.cc:2781] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:14.655712 17203 leader_election.cc:302] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e; no voters: cdccadabd9154ea69e15405d75301fc1
I20240501 08:04:14.655934 17351 leader_election.cc:302] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:14.654925 17760 raft_consensus.cc:686] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:14.658082 17466 raft_consensus.cc:686] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:14.658308 17484 raft_consensus.cc:2781] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:04:14.659754 17484 raft_consensus.cc:686] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:14.658875 17466 consensus_queue.cc:237] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.663956 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee19995987e4749b8c949a60e1ae1e9" candidate_uuid: "fa9d3bc74a68481e9de647223cb0110e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
W20240501 08:04:14.666963 17203 leader_election.cc:341] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:14.668181 17351 leader_election.cc:302] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:14.655002 17807 raft_consensus.cc:686] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Becoming Leader. State: Replica: fa9d3bc74a68481e9de647223cb0110e, State: Running, Role: LEADER
I20240501 08:04:14.660483 17484 consensus_queue.cc:237] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.661402 17713 raft_consensus.cc:3037] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.677985 17788 raft_consensus.cc:2726] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:14.692185 17428 tablet_bootstrap.cc:492] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:14.659293 17760 consensus_queue.cc:237] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.692783 17428 ts_tablet_manager.cc:1367] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.359s user 0.007s sys 0.023s
I20240501 08:04:14.689575 17807 consensus_queue.cc:237] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.697286 17428 raft_consensus.cc:348] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.698091 17428 raft_consensus.cc:374] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.698407 17428 raft_consensus.cc:729] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.711723 17428 consensus_queue.cc:260] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.709609 17132 catalog_manager.cc:5484] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:14.722379 17428 ts_tablet_manager.cc:1398] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20240501 08:04:14.727027 17713 raft_consensus.cc:2726] T 7a64317c43cd4a2598c392879c3ff65a P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:14.723186 17428 tablet_bootstrap.cc:492] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:14.799909 17432 raft_consensus.cc:482] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.814018 17432 raft_consensus.cc:504] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.739069 17135 catalog_manager.cc:5484] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
W20240501 08:04:14.824792 17830 log.cc:924] Time spent T 7a64317c43cd4a2598c392879c3ff65a P fa9d3bc74a68481e9de647223cb0110e: Append to log took a long time: real 0.087s user 0.001s sys 0.000s
I20240501 08:04:14.828990 17380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:14.829851 17380 raft_consensus.cc:2379] T 78bfdb70077142128e0514f841189ca5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdccadabd9154ea69e15405d75301fc1 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:14.830824 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78bfdb70077142128e0514f841189ca5" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:14.836800 17277 leader_election.cc:302] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1; no voters: 5655c2e9168d4eeaa212db8874c2efa5, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:14.839010 17755 raft_consensus.cc:3037] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:14.840062 17432 leader_election.cc:288] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
W20240501 08:04:14.846074 17828 log.cc:924] Time spent T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e: Append to log took a long time: real 0.119s user 0.000s sys 0.000s
I20240501 08:04:14.733634 17760 consensus_queue.cc:1035] T 7a64317c43cd4a2598c392879c3ff65a P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.108s
I20240501 08:04:14.852384 17428 tablet_bootstrap.cc:654] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:14.852381 17426 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: No bootstrap required, opened a new log
I20240501 08:04:14.853467 17426 ts_tablet_manager.cc:1367] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: Time spent bootstrapping tablet: real 0.315s user 0.014s sys 0.004s
I20240501 08:04:14.857357 17426 raft_consensus.cc:348] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.859289 17426 raft_consensus.cc:374] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.859692 17426 raft_consensus.cc:729] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.860270 17426 consensus_queue.cc:260] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.864924 17755 raft_consensus.cc:2726] T 78bfdb70077142128e0514f841189ca5 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:14.866456 17426 ts_tablet_manager.cc:1398] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20240501 08:04:14.869956 17428 tablet_bootstrap.cc:492] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:14.870378 17428 ts_tablet_manager.cc:1367] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.147s user 0.051s sys 0.042s
I20240501 08:04:14.872907 17428 raft_consensus.cc:348] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.873657 17428 raft_consensus.cc:374] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:14.873893 17428 raft_consensus.cc:729] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:14.874455 17428 consensus_queue.cc:260] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.876444 17428 ts_tablet_manager.cc:1398] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 08:04:14.877270 17428 tablet_bootstrap.cc:492] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:14.931918 17132 catalog_manager.cc:5484] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:14.935400 17132 catalog_manager.cc:5484] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:14.945639 17132 catalog_manager.cc:5484] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa9d3bc74a68481e9de647223cb0110e (127.15.168.129). New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:14.947285 17712 raft_consensus.cc:482] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:14.948309 17712 raft_consensus.cc:504] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:14.968590 17712 leader_election.cc:288] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:14.981786 17232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:14.983342 17232 raft_consensus.cc:2379] T e93329da82834bd19fd547b7a5abdadb P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:14.984824 17428 tablet_bootstrap.cc:654] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
W20240501 08:04:14.990795 17278 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.131:43915, user_credentials={real_user=slave}} blocked reactor thread for 43749.6us
I20240501 08:04:14.994765 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93329da82834bd19fd547b7a5abdadb" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:15.012812 17428 tablet_bootstrap.cc:492] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:15.013607 17428 ts_tablet_manager.cc:1367] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.137s user 0.009s sys 0.011s
I20240501 08:04:15.018149 17428 raft_consensus.cc:348] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.018922 17428 raft_consensus.cc:374] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:15.019289 17428 raft_consensus.cc:729] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:15.019968 17428 consensus_queue.cc:260] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.020680 17351 leader_election.cc:302] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:15.023836 17840 raft_consensus.cc:3037] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.037130 17840 raft_consensus.cc:2726] T e93329da82834bd19fd547b7a5abdadb P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:15.038982 17428 ts_tablet_manager.cc:1398] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.025s user 0.001s sys 0.006s
I20240501 08:04:15.040843 17428 tablet_bootstrap.cc:492] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:15.046447 17428 tablet_bootstrap.cc:654] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:15.048592 17432 raft_consensus.cc:482] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:15.049131 17432 raft_consensus.cc:504] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.051236 17432 leader_election.cc:288] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:15.054365 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:15.055152 17233 raft_consensus.cc:2454] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 0.
I20240501 08:04:15.056263 17277 leader_election.cc:302] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:15.057293 17755 raft_consensus.cc:2781] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:04:15.057678 17755 raft_consensus.cc:482] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:04:15.057989 17755 raft_consensus.cc:3037] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.058277 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5" is_pre_election: true
I20240501 08:04:15.061779 17428 tablet_bootstrap.cc:492] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:15.062222 17428 ts_tablet_manager.cc:1367] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.009s
W20240501 08:04:15.064306 17278 leader_election.cc:341] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:15.064586 17428 raft_consensus.cc:348] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.065276 17428 raft_consensus.cc:374] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:15.065016 17755 raft_consensus.cc:504] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.065546 17428 raft_consensus.cc:729] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:15.066205 17428 consensus_queue.cc:260] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.070905 17428 ts_tablet_manager.cc:1398] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20240501 08:04:15.072057 17428 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5: Bootstrap starting.
I20240501 08:04:15.072677 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e"
I20240501 08:04:15.072783 17755 leader_election.cc:288] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Requested vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:15.073230 17233 raft_consensus.cc:3037] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.078277 17428 tablet_bootstrap.cc:654] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5: Neither blocks nor log segments found. Creating new log.
I20240501 08:04:15.079460 17233 raft_consensus.cc:2454] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdccadabd9154ea69e15405d75301fc1 in term 1.
I20240501 08:04:15.080355 17277 leader_election.cc:302] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e; no voters:
I20240501 08:04:15.080931 17381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "cdccadabd9154ea69e15405d75301fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655c2e9168d4eeaa212db8874c2efa5"
I20240501 08:04:15.081512 17755 raft_consensus.cc:2781] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader election won for term 1
W20240501 08:04:15.082270 17278 leader_election.cc:341] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:04:15.082301 17755 raft_consensus.cc:686] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Becoming Leader. State: Replica: cdccadabd9154ea69e15405d75301fc1, State: Running, Role: LEADER
I20240501 08:04:15.083470 17755 consensus_queue.cc:237] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.089481 17428 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5: No bootstrap required, opened a new log
I20240501 08:04:15.089920 17428 ts_tablet_manager.cc:1367] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240501 08:04:15.091965 17837 raft_consensus.cc:482] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:15.092193 17428 raft_consensus.cc:348] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.092433 17837 raft_consensus.cc:504] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.093075 17428 raft_consensus.cc:374] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:04:15.093380 17428 raft_consensus.cc:729] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655c2e9168d4eeaa212db8874c2efa5, State: Initialized, Role: FOLLOWER
I20240501 08:04:15.093959 17428 consensus_queue.cc:260] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.094504 17837 leader_election.cc:288] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:15.095376 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee19995987e4749b8c949a60e1ae1e9" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:15.095871 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee19995987e4749b8c949a60e1ae1e9" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:15.096228 17310 raft_consensus.cc:2379] T 0ee19995987e4749b8c949a60e1ae1e9 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:15.098299 17428 ts_tablet_manager.cc:1398] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20240501 08:04:15.100836 17840 raft_consensus.cc:482] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:15.101395 17840 raft_consensus.cc:504] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.103823 17351 leader_election.cc:302] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:15.104044 17840 leader_election.cc:288] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:15.104679 17791 raft_consensus.cc:3037] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.107021 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3abec6d380a4a5fa72fd34fe37bbd92" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:15.107908 17310 raft_consensus.cc:2379] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate fa9d3bc74a68481e9de647223cb0110e in this term.
I20240501 08:04:15.107424 17135 catalog_manager.cc:5484] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdccadabd9154ea69e15405d75301fc1 (127.15.168.130). New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:15.111644 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3abec6d380a4a5fa72fd34fe37bbd92" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:15.111814 17791 raft_consensus.cc:2726] T 0ee19995987e4749b8c949a60e1ae1e9 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:15.113162 17351 leader_election.cc:302] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:15.113811 17840 raft_consensus.cc:3037] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.119330 17840 raft_consensus.cc:2726] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:15.128274 17619 consensus_queue.cc:1035] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.141175 17840 raft_consensus.cc:482] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:15.141772 17840 raft_consensus.cc:504] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.145431 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:15.145833 17310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c8eff63a7449018719d17f779939c5" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:15.147202 17840 leader_election.cc:288] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:15.147468 17351 leader_election.cc:302] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:15.149106 17619 consensus_queue.cc:1035] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.148844 17840 raft_consensus.cc:2726] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:15.155609 17310 raft_consensus.cc:3037] T a0fe90f0956c48d184cd48f76c7e2a64 P cdccadabd9154ea69e15405d75301fc1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.159904 17619 consensus_queue.cc:1035] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:15.160187 17755 consensus_queue.cc:1035] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:15.169337 17839 consensus_queue.cc:1035] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.170131 17619 consensus_queue.cc:1035] T a0fe90f0956c48d184cd48f76c7e2a64 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.175637 17383 raft_consensus.cc:3037] T 96c8eff63a7449018719d17f779939c5 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.191262 17755 consensus_queue.cc:1035] T e93329da82834bd19fd547b7a5abdadb P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:15.230012 17839 consensus_queue.cc:1035] T 0ee19995987e4749b8c949a60e1ae1e9 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:15.242874 17663 consensus_queue.cc:1035] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.248909 17838 consensus_queue.cc:1035] T 78bfdb70077142128e0514f841189ca5 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.405566 16034 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:04:15.420163 17856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:15.448194 17859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:04:15.449120 17857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:04:15.450088 16034 server_base.cc:1034] running on GCE node
I20240501 08:04:15.451342 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240501 08:04:15.451591 16034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240501 08:04:15.451798 16034 hybrid_clock.cc:648] HybridClock initialized: now 1714550655451780 us; error 0 us; skew 500 ppm
I20240501 08:04:15.452486 16034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240501 08:04:15.459931 16034 webserver.cc:460] Webserver started at http://127.15.168.132:34733/ using document root <none> and password file <none>
I20240501 08:04:15.460603 16034 fs_manager.cc:359] Metadata directory not provided
I20240501 08:04:15.460835 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:04:15.461143 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:04:15.462554 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-3-root/instance:
uuid: "010ae6a71a524121b75804307f9645f9"
format_stamp: "Formatted at 2024-05-01 08:04:15 on dist-test-slave-g8jl"
I20240501 08:04:15.467873 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20240501 08:04:15.471637 17864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:15.472510 16034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20240501 08:04:15.472862 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-3-root
uuid: "010ae6a71a524121b75804307f9645f9"
format_stamp: "Formatted at 2024-05-01 08:04:15 on dist-test-slave-g8jl"
I20240501 08:04:15.473191 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:04:15.503908 16034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:04:15.505374 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:04:15.507433 16034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:04:15.509904 16034 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:04:15.510110 16034 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:15.510393 16034 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:04:15.510601 16034 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:04:15.513788 17837 raft_consensus.cc:482] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:04:15.514307 17837 raft_consensus.cc:504] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } }
I20240501 08:04:15.516592 17837 leader_election.cc:288] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541), cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:15.517877 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdccadabd9154ea69e15405d75301fc1" is_pre_election: true
I20240501 08:04:15.518383 17230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd19ca98cfd4974ad3fca19eb088cc0" candidate_uuid: "5655c2e9168d4eeaa212db8874c2efa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa9d3bc74a68481e9de647223cb0110e" is_pre_election: true
I20240501 08:04:15.519229 17230 raft_consensus.cc:2379] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5655c2e9168d4eeaa212db8874c2efa5 in current term 1: Already voted for candidate cdccadabd9154ea69e15405d75301fc1 in this term.
I20240501 08:04:15.520591 17351 leader_election.cc:302] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5655c2e9168d4eeaa212db8874c2efa5; no voters: cdccadabd9154ea69e15405d75301fc1, fa9d3bc74a68481e9de647223cb0110e
I20240501 08:04:15.521987 17837 raft_consensus.cc:3037] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:04:15.529280 17837 raft_consensus.cc:2726] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:04:15.553650 17432 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:04:15.579720 17621 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240501 08:04:15.634893 16034 rpc_server.cc:307] RPC server started. Bound to: 127.15.168.132:45715
I20240501 08:04:15.635418 17928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.168.132:45715 every 8 connection(s)
I20240501 08:04:15.639873 16034 auto_leader_rebalancer-test.cc:405] add a tserver: 010ae6a71a524121b75804307f9645f9
I20240501 08:04:15.655596 17929 heartbeater.cc:347] Connected to a master server at 127.15.168.190:41731
I20240501 08:04:15.656040 17929 heartbeater.cc:464] Registering TS with master...
I20240501 08:04:15.656885 17929 heartbeater.cc:511] Master 127.15.168.190:41731 requested a full tablet report, sending...
I20240501 08:04:15.658730 17135 ts_manager.cc:194] Registered new tserver with Master: 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715)
I20240501 08:04:15.660010 17135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58944
I20240501 08:04:16.418481 17306 consensus_queue.cc:237] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:16.424620 17381 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.425714 17233 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.425997 17621 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:16.428525 17621 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:16.433982 17755 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.437227 17233 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
W20240501 08:04:16.438807 17277 consensus_peers.cc:485] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cd19ca98cfd4974ad3fca19eb088cc0. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:16.439343 17381 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.456128 17135 catalog_manager.cc:5484] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:16.483315 17233 consensus_queue.cc:237] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:16.491993 17381 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.493022 17619 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:16.494041 17306 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.498394 17619 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240501 08:04:16.506752 17202 consensus_peers.cc:485] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e00624b7db1409890298f5a6d88a575. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:16.515846 17939 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.518361 17381 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.529109 17233 consensus_queue.cc:237] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:16.530709 17306 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.545166 17381 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.545765 17132 catalog_manager.cc:5484] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.546499 17310 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.547699 17619 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:16.548601 17939 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240501 08:04:16.552110 17202 consensus_peers.cc:485] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2250a60c258544668fa164fcf11634e4. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:16.560531 17939 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.562052 17306 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.580845 17134 catalog_manager.cc:5484] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:16.580454 17381 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.597978 17381 consensus_queue.cc:237] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:16.616989 17306 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.617359 17234 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:16.618620 17837 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:16.619586 17952 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20240501 08:04:16.628413 17351 consensus_peers.cc:485] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1189be88e7b341918c1fa148b2d77030. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:16.633510 17952 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.644973 17234 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.652114 17306 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:16.655041 17134 catalog_manager.cc:5484] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:16.663286 17929 heartbeater.cc:503] Master 127.15.168.190:41731 was elected leader, sending a full tablet report...
I20240501 08:04:16.836190 17957 ts_tablet_manager.cc:902] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:16.842756 17957 tablet_copy_client.cc:308] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.130:33757
I20240501 08:04:16.865777 17322 tablet_copy_service.cc:140] P cdccadabd9154ea69e15405d75301fc1: Received BeginTabletCopySession request for tablet 9cd19ca98cfd4974ad3fca19eb088cc0 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:50154)
I20240501 08:04:16.866407 17322 tablet_copy_service.cc:161] P cdccadabd9154ea69e15405d75301fc1: Beginning new tablet copy session on tablet 9cd19ca98cfd4974ad3fca19eb088cc0 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:50154: session id = 010ae6a71a524121b75804307f9645f9-9cd19ca98cfd4974ad3fca19eb088cc0
I20240501 08:04:16.880034 17322 tablet_copy_source_session.cc:205] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:16.884670 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd19ca98cfd4974ad3fca19eb088cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:16.898562 17957 tablet_copy_client.cc:783] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:16.899142 17957 tablet_copy_client.cc:647] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:16.904848 17957 tablet_copy_client.cc:523] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:16.912364 17957 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:16.930858 17957 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:16.931772 17957 tablet_bootstrap.cc:492] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:16.932384 17957 ts_tablet_manager.cc:1367] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.020s user 0.015s sys 0.004s
I20240501 08:04:16.934717 17957 raft_consensus.cc:348] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:16.935530 17957 raft_consensus.cc:729] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:16.936136 17957 consensus_queue.cc:260] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:16.939028 17957 ts_tablet_manager.cc:1398] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.006s user 0.006s sys 0.003s
I20240501 08:04:16.940979 17322 tablet_copy_service.cc:342] P cdccadabd9154ea69e15405d75301fc1: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-9cd19ca98cfd4974ad3fca19eb088cc0 received from {username='slave'} at 127.0.0.1:50154
I20240501 08:04:16.941423 17322 tablet_copy_service.cc:433] P cdccadabd9154ea69e15405d75301fc1: ending tablet copy session 010ae6a71a524121b75804307f9645f9-9cd19ca98cfd4974ad3fca19eb088cc0 on tablet 9cd19ca98cfd4974ad3fca19eb088cc0 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:16.969012 17957 ts_tablet_manager.cc:902] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:16.982044 17957 tablet_copy_client.cc:308] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.129:44541
I20240501 08:04:16.993882 17247 tablet_copy_service.cc:140] P fa9d3bc74a68481e9de647223cb0110e: Received BeginTabletCopySession request for tablet 3e00624b7db1409890298f5a6d88a575 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:36634)
I20240501 08:04:16.994472 17247 tablet_copy_service.cc:161] P fa9d3bc74a68481e9de647223cb0110e: Beginning new tablet copy session on tablet 3e00624b7db1409890298f5a6d88a575 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:36634: session id = 010ae6a71a524121b75804307f9645f9-3e00624b7db1409890298f5a6d88a575
I20240501 08:04:17.001259 17247 tablet_copy_source_session.cc:205] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:17.016026 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e00624b7db1409890298f5a6d88a575. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:17.028004 17957 tablet_copy_client.cc:783] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:17.028537 17957 tablet_copy_client.cc:647] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:17.032320 17957 tablet_copy_client.cc:523] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:17.038381 17957 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:17.048156 17971 ts_tablet_manager.cc:902] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:17.067826 17970 ts_tablet_manager.cc:902] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:17.077296 17957 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:17.078133 17957 tablet_bootstrap.cc:492] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:17.078775 17957 ts_tablet_manager.cc:1367] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.041s user 0.034s sys 0.004s
I20240501 08:04:17.081323 17957 raft_consensus.cc:348] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:17.082170 17957 raft_consensus.cc:729] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:17.082779 17957 consensus_queue.cc:260] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:17.092432 17957 ts_tablet_manager.cc:1398] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20240501 08:04:17.094362 17970 tablet_copy_client.cc:308] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.129:44541
I20240501 08:04:17.096465 17247 tablet_copy_service.cc:140] P fa9d3bc74a68481e9de647223cb0110e: Received BeginTabletCopySession request for tablet 2250a60c258544668fa164fcf11634e4 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:36634)
I20240501 08:04:17.096954 17247 tablet_copy_service.cc:161] P fa9d3bc74a68481e9de647223cb0110e: Beginning new tablet copy session on tablet 2250a60c258544668fa164fcf11634e4 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:36634: session id = 010ae6a71a524121b75804307f9645f9-2250a60c258544668fa164fcf11634e4
I20240501 08:04:17.098841 17971 tablet_copy_client.cc:308] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.131:43915
I20240501 08:04:17.099463 17246 tablet_copy_service.cc:342] P fa9d3bc74a68481e9de647223cb0110e: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-3e00624b7db1409890298f5a6d88a575 received from {username='slave'} at 127.0.0.1:36634
I20240501 08:04:17.099880 17246 tablet_copy_service.cc:433] P fa9d3bc74a68481e9de647223cb0110e: ending tablet copy session 010ae6a71a524121b75804307f9645f9-3e00624b7db1409890298f5a6d88a575 on tablet 3e00624b7db1409890298f5a6d88a575 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:17.108549 17247 tablet_copy_source_session.cc:205] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:17.112200 17970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2250a60c258544668fa164fcf11634e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:17.122241 17396 tablet_copy_service.cc:140] P 5655c2e9168d4eeaa212db8874c2efa5: Received BeginTabletCopySession request for tablet 1189be88e7b341918c1fa148b2d77030 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:44248)
I20240501 08:04:17.122833 17396 tablet_copy_service.cc:161] P 5655c2e9168d4eeaa212db8874c2efa5: Beginning new tablet copy session on tablet 1189be88e7b341918c1fa148b2d77030 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:44248: session id = 010ae6a71a524121b75804307f9645f9-1189be88e7b341918c1fa148b2d77030
I20240501 08:04:17.130316 17970 tablet_copy_client.cc:783] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:17.131738 17396 tablet_copy_source_session.cc:205] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:17.132442 17970 tablet_copy_client.cc:647] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:17.134682 17971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1189be88e7b341918c1fa148b2d77030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:17.145044 17970 tablet_copy_client.cc:523] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:17.147861 17971 tablet_copy_client.cc:783] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:17.148375 17971 tablet_copy_client.cc:647] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:17.151696 17971 tablet_copy_client.cc:523] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:17.155593 17970 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:17.159555 17971 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:17.181906 17970 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:17.182701 17970 tablet_bootstrap.cc:492] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:17.183323 17970 ts_tablet_manager.cc:1367] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.028s user 0.017s sys 0.004s
I20240501 08:04:17.186036 17970 raft_consensus.cc:348] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:17.186877 17970 raft_consensus.cc:729] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:17.187709 17971 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:17.187453 17970 consensus_queue.cc:260] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:17.189611 17970 ts_tablet_manager.cc:1398] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20240501 08:04:17.190351 17971 tablet_bootstrap.cc:492] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:17.192176 17247 tablet_copy_service.cc:342] P fa9d3bc74a68481e9de647223cb0110e: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-2250a60c258544668fa164fcf11634e4 received from {username='slave'} at 127.0.0.1:36634
I20240501 08:04:17.192508 17247 tablet_copy_service.cc:433] P fa9d3bc74a68481e9de647223cb0110e: ending tablet copy session 010ae6a71a524121b75804307f9645f9-2250a60c258544668fa164fcf11634e4 on tablet 2250a60c258544668fa164fcf11634e4 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:17.191826 17971 ts_tablet_manager.cc:1367] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.010s
I20240501 08:04:17.199965 17971 raft_consensus.cc:348] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:17.200688 17971 raft_consensus.cc:729] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:17.201229 17971 consensus_queue.cc:260] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:17.203366 17971 ts_tablet_manager.cc:1398] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20240501 08:04:17.204764 17396 tablet_copy_service.cc:342] P 5655c2e9168d4eeaa212db8874c2efa5: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-1189be88e7b341918c1fa148b2d77030 received from {username='slave'} at 127.0.0.1:44248
I20240501 08:04:17.205056 17396 tablet_copy_service.cc:433] P 5655c2e9168d4eeaa212db8874c2efa5: ending tablet copy session 010ae6a71a524121b75804307f9645f9-1189be88e7b341918c1fa148b2d77030 on tablet 1189be88e7b341918c1fa148b2d77030 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:17.322113 17903 raft_consensus.cc:1203] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:17.458304 17903 raft_consensus.cc:1203] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:17.494387 17903 raft_consensus.cc:1203] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:17.554842 17903 raft_consensus.cc:1203] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:17.881778 17755 raft_consensus.cc:1050] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:17.884177 17755 consensus_queue.cc:237] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:17.889986 17381 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.890081 17903 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.892005 17621 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:17.894908 17987 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:17.904793 17233 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.906977 17986 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:17.914587 17986 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:17.915822 17381 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:17.927644 17230 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:17.931116 17903 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:17.935875 17134 catalog_manager.cc:5484] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:04:17.938097 17939 raft_consensus.cc:1050] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:17.946040 17939 consensus_queue.cc:237] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:17.949232 17984 raft_consensus.cc:1050] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:17.951611 17984 consensus_queue.cc:237] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:17.969813 17903 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.970261 17381 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.971261 17816 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:17.972239 17619 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:17.985277 17903 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.987181 17998 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:04:17.988662 17381 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.988904 17310 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:17.990779 17998 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:17.997107 17984 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:04:18.010879 17984 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.012004 17381 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
W20240501 08:04:18.023147 17351 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.168.130:33757, user_credentials={real_user=slave}} blocked reactor thread for 48663.1us
I20240501 08:04:18.032804 17307 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.045600 17903 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.042268 17619 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.056119 17307 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:18.057003 17382 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.060788 17939 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:18.074069 17903 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.077625 17837 raft_consensus.cc:1050] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:18.083204 17837 consensus_queue.cc:237] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:18.084424 17310 consensus_queue.cc:237] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:18.095857 17307 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.102886 17132 catalog_manager.cc:5484] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.104487 17132 catalog_manager.cc:5484] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.112996 17305 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:18.113440 17233 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:18.117357 17902 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.116062 17997 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20240501 08:04:18.118898 17986 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:18.119486 17840 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:18.115105 17903 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:18.132179 17621 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.134506 17230 consensus_queue.cc:237] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:18.139828 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9cd19ca98cfd4974ad3fca19eb088cc0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:18.141168 17381 raft_consensus.cc:1261] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.140575 17903 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.142774 17986 consensus_queue.cc:1035] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:18.143595 17902 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.145088 17994 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240501 08:04:18.142946 17840 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20240501 08:04:18.146135 17381 raft_consensus.cc:1261] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.147672 17385 raft_consensus.cc:2932] T 9cd19ca98cfd4974ad3fca19eb088cc0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.152634 17232 consensus_queue.cc:237] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:18.155624 17901 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.153832 17984 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.165393 17135 catalog_manager.cc:5484] T 9cd19ca98cfd4974ad3fca19eb088cc0 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:04:18.167848 17900 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.169276 17816 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:18.171422 17310 raft_consensus.cc:1261] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.172477 17816 consensus_queue.cc:1035] T 3e00624b7db1409890298f5a6d88a575 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:18.176530 17998 consensus_queue.cc:1035] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:18.177587 17939 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.178644 17900 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.186564 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2250a60c258544668fa164fcf11634e4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:18.185256 17383 raft_consensus.cc:2932] T 3e00624b7db1409890298f5a6d88a575 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.193370 17226 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd19ca98cfd4974ad3fca19eb088cc0 with delete_type TABLET_DATA_TOMBSTONED (TS fa9d3bc74a68481e9de647223cb0110e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52016
I20240501 08:04:18.198918 17134 catalog_manager.cc:5484] T 3e00624b7db1409890298f5a6d88a575 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.207410 18010 tablet_replica.cc:330] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: stopping tablet replica
I20240501 08:04:18.208351 18010 raft_consensus.cc:2229] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:18.208845 18010 raft_consensus.cc:2258] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:18.213457 17997 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.214195 17135 catalog_manager.cc:5484] T 2250a60c258544668fa164fcf11634e4 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:04:18.221954 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3e00624b7db1409890298f5a6d88a575 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:18.226446 17305 raft_consensus.cc:2932] T 2250a60c258544668fa164fcf11634e4 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.227433 17376 tablet_service.cc:1514] Processing DeleteTablet for tablet 2250a60c258544668fa164fcf11634e4 with delete_type TABLET_DATA_TOMBSTONED (TS 5655c2e9168d4eeaa212db8874c2efa5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37214
I20240501 08:04:18.239310 17307 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.249186 17230 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.268168 17295 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e00624b7db1409890298f5a6d88a575 with delete_type TABLET_DATA_TOMBSTONED (TS cdccadabd9154ea69e15405d75301fc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55168
I20240501 08:04:18.232846 18010 ts_tablet_manager.cc:1875] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:18.293295 17133 catalog_manager.cc:5484] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.285882 17902 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.331064 18014 tablet_replica.cc:330] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: stopping tablet replica
I20240501 08:04:18.332118 18014 raft_consensus.cc:2229] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:18.332741 18014 raft_consensus.cc:2258] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:18.336377 18012 tablet_replica.cc:330] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: stopping tablet replica
I20240501 08:04:18.337677 18012 raft_consensus.cc:2229] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:18.352087 18012 raft_consensus.cc:2258] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:18.354189 18014 ts_tablet_manager.cc:1875] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:18.358690 18012 ts_tablet_manager.cc:1875] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:18.394322 17383 consensus_queue.cc:237] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:18.398325 18010 ts_tablet_manager.cc:1888] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:18.398743 18010 log.cc:1196] T 9cd19ca98cfd4974ad3fca19eb088cc0 P fa9d3bc74a68481e9de647223cb0110e: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root/wals/9cd19ca98cfd4974ad3fca19eb088cc0
I20240501 08:04:18.400527 18012 ts_tablet_manager.cc:1888] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:18.400965 18012 log.cc:1196] T 2250a60c258544668fa164fcf11634e4 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root/wals/2250a60c258544668fa164fcf11634e4
I20240501 08:04:18.401192 17121 catalog_manager.cc:4830] TS fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): tablet 9cd19ca98cfd4974ad3fca19eb088cc0 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:18.403698 17902 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.404803 17837 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:18.406903 17305 raft_consensus.cc:1261] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:18.410688 18014 ts_tablet_manager.cc:1888] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:18.411145 18014 log.cc:1196] T 3e00624b7db1409890298f5a6d88a575 P cdccadabd9154ea69e15405d75301fc1: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root/wals/3e00624b7db1409890298f5a6d88a575
I20240501 08:04:18.412541 17121 catalog_manager.cc:4830] TS cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): tablet 3e00624b7db1409890298f5a6d88a575 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:18.413709 17122 catalog_manager.cc:4830] TS 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): tablet 2250a60c258544668fa164fcf11634e4 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:18.413158 17837 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.414542 17902 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.416960 18013 consensus_queue.cc:1035] T 1189be88e7b341918c1fa148b2d77030 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240501 08:04:18.421365 17310 raft_consensus.cc:2932] T 1189be88e7b341918c1fa148b2d77030 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.427639 17122 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1189be88e7b341918c1fa148b2d77030 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:18.430509 17135 catalog_manager.cc:5484] T 1189be88e7b341918c1fa148b2d77030 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:18.439998 17226 tablet_service.cc:1514] Processing DeleteTablet for tablet 1189be88e7b341918c1fa148b2d77030 with delete_type TABLET_DATA_TOMBSTONED (TS fa9d3bc74a68481e9de647223cb0110e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52016
I20240501 08:04:18.441598 18010 tablet_replica.cc:330] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: stopping tablet replica
I20240501 08:04:18.445135 18010 raft_consensus.cc:2229] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:18.445731 18010 raft_consensus.cc:2258] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:18.449005 18010 ts_tablet_manager.cc:1875] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:18.460033 18010 ts_tablet_manager.cc:1888] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:18.460379 18010 log.cc:1196] T 1189be88e7b341918c1fa148b2d77030 P fa9d3bc74a68481e9de647223cb0110e: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root/wals/1189be88e7b341918c1fa148b2d77030
I20240501 08:04:18.461622 17121 catalog_manager.cc:4830] TS fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): tablet 1189be88e7b341918c1fa148b2d77030 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:19.496170 17232 consensus_queue.cc:237] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:19.502651 17307 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240501 08:04:19.502844 17202 consensus_peers.cc:485] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e4b47c5f05da46bc9d9b5b4b202c1210. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:19.503244 17382 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:19.504725 18017 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.505458 17994 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.516036 17939 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.519075 17382 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.526077 17307 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.534209 17382 consensus_queue.cc:237] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:19.539424 17133 catalog_manager.cc:5484] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.553279 17305 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:19.553591 17232 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240501 08:04:19.554576 17351 consensus_peers.cc:485] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66624ff54d8844e3bf88af88be895fd3. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:19.556017 18013 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.557408 17997 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.572273 17837 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.574314 17305 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.585821 17233 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.590947 17382 consensus_queue.cc:237] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:19.595378 17135 catalog_manager.cc:5484] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 08:04:19.604506 17351 consensus_peers.cc:485] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5f504d6ad6b4b77927fb216e12347e8. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:19.606684 17305 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:19.608738 17837 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.609172 17233 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:19.610994 18013 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.619999 18013 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.622490 17307 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.627822 17232 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.632164 17382 consensus_queue.cc:237] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:19.643735 17305 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240501 08:04:19.644032 17351 consensus_peers.cc:485] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 068fca00dea44c6fbac5bc5365a66a4b. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:19.645141 17997 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.647723 17133 catalog_manager.cc:5484] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.651362 17233 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:19.652660 17837 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:19.661043 18031 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.664332 17305 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.662799 17233 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:19.691593 17132 catalog_manager.cc:5484] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:20.010645 18035 ts_tablet_manager.cc:902] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:20.019147 18035 tablet_copy_client.cc:308] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.131:43915
I20240501 08:04:20.019958 18040 ts_tablet_manager.cc:902] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:20.020830 17396 tablet_copy_service.cc:140] P 5655c2e9168d4eeaa212db8874c2efa5: Received BeginTabletCopySession request for tablet 66624ff54d8844e3bf88af88be895fd3 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:44248)
I20240501 08:04:20.021361 17396 tablet_copy_service.cc:161] P 5655c2e9168d4eeaa212db8874c2efa5: Beginning new tablet copy session on tablet 66624ff54d8844e3bf88af88be895fd3 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:44248: session id = 010ae6a71a524121b75804307f9645f9-66624ff54d8844e3bf88af88be895fd3
I20240501 08:04:20.023962 18040 tablet_copy_client.cc:308] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.129:44541
I20240501 08:04:20.025318 17247 tablet_copy_service.cc:140] P fa9d3bc74a68481e9de647223cb0110e: Received BeginTabletCopySession request for tablet e4b47c5f05da46bc9d9b5b4b202c1210 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:36634)
I20240501 08:04:20.025840 17247 tablet_copy_service.cc:161] P fa9d3bc74a68481e9de647223cb0110e: Beginning new tablet copy session on tablet e4b47c5f05da46bc9d9b5b4b202c1210 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:36634: session id = 010ae6a71a524121b75804307f9645f9-e4b47c5f05da46bc9d9b5b4b202c1210
I20240501 08:04:20.027383 17396 tablet_copy_source_session.cc:205] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:20.031601 18035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66624ff54d8844e3bf88af88be895fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:20.032115 17247 tablet_copy_source_session.cc:205] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:20.037039 18040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b47c5f05da46bc9d9b5b4b202c1210. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:20.046057 18035 tablet_copy_client.cc:783] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:20.046679 18035 tablet_copy_client.cc:647] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:20.050419 18040 tablet_copy_client.cc:783] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:20.051070 18040 tablet_copy_client.cc:647] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:20.051602 18035 tablet_copy_client.cc:523] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:20.054267 18040 tablet_copy_client.cc:523] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:20.060325 18045 ts_tablet_manager.cc:902] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:20.061039 18035 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:20.067672 18040 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:20.087574 18035 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:20.088567 18035 tablet_bootstrap.cc:492] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:20.089289 18035 ts_tablet_manager.cc:1367] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.028s user 0.025s sys 0.004s
I20240501 08:04:20.093228 18035 raft_consensus.cc:348] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.094151 18035 raft_consensus.cc:729] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:20.094810 18035 consensus_queue.cc:260] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.104620 18045 tablet_copy_client.cc:308] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.131:43915
I20240501 08:04:20.106824 17396 tablet_copy_service.cc:140] P 5655c2e9168d4eeaa212db8874c2efa5: Received BeginTabletCopySession request for tablet e5f504d6ad6b4b77927fb216e12347e8 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:44248)
I20240501 08:04:20.107365 17396 tablet_copy_service.cc:161] P 5655c2e9168d4eeaa212db8874c2efa5: Beginning new tablet copy session on tablet e5f504d6ad6b4b77927fb216e12347e8 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:44248: session id = 010ae6a71a524121b75804307f9645f9-e5f504d6ad6b4b77927fb216e12347e8
I20240501 08:04:20.107458 18035 ts_tablet_manager.cc:1398] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.018s user 0.011s sys 0.003s
I20240501 08:04:20.115612 17395 tablet_copy_service.cc:342] P 5655c2e9168d4eeaa212db8874c2efa5: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-66624ff54d8844e3bf88af88be895fd3 received from {username='slave'} at 127.0.0.1:44248
I20240501 08:04:20.116070 17395 tablet_copy_service.cc:433] P 5655c2e9168d4eeaa212db8874c2efa5: ending tablet copy session 010ae6a71a524121b75804307f9645f9-66624ff54d8844e3bf88af88be895fd3 on tablet 66624ff54d8844e3bf88af88be895fd3 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:20.116977 17396 tablet_copy_source_session.cc:205] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:20.121358 18045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f504d6ad6b4b77927fb216e12347e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:20.124270 18040 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:20.125281 18040 tablet_bootstrap.cc:492] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:20.126127 18035 ts_tablet_manager.cc:902] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:20.126854 18040 ts_tablet_manager.cc:1367] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.059s user 0.021s sys 0.001s
I20240501 08:04:20.130949 18040 raft_consensus.cc:348] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.131845 18040 raft_consensus.cc:729] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:20.132524 18040 consensus_queue.cc:260] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.136433 18040 ts_tablet_manager.cc:1398] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240501 08:04:20.138063 17247 tablet_copy_service.cc:342] P fa9d3bc74a68481e9de647223cb0110e: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-e4b47c5f05da46bc9d9b5b4b202c1210 received from {username='slave'} at 127.0.0.1:36634
I20240501 08:04:20.138484 17247 tablet_copy_service.cc:433] P fa9d3bc74a68481e9de647223cb0110e: ending tablet copy session 010ae6a71a524121b75804307f9645f9-e4b47c5f05da46bc9d9b5b4b202c1210 on tablet e4b47c5f05da46bc9d9b5b4b202c1210 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:20.148137 18045 tablet_copy_client.cc:783] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:20.148785 18045 tablet_copy_client.cc:647] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:20.154104 18045 tablet_copy_client.cc:523] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:20.167161 18045 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:20.183414 18035 tablet_copy_client.cc:308] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.131:43915
I20240501 08:04:20.184769 17396 tablet_copy_service.cc:140] P 5655c2e9168d4eeaa212db8874c2efa5: Received BeginTabletCopySession request for tablet 068fca00dea44c6fbac5bc5365a66a4b from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:44248)
I20240501 08:04:20.185288 17396 tablet_copy_service.cc:161] P 5655c2e9168d4eeaa212db8874c2efa5: Beginning new tablet copy session on tablet 068fca00dea44c6fbac5bc5365a66a4b from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:44248: session id = 010ae6a71a524121b75804307f9645f9-068fca00dea44c6fbac5bc5365a66a4b
I20240501 08:04:20.187178 18045 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:20.188287 18045 tablet_bootstrap.cc:492] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:20.188933 18045 ts_tablet_manager.cc:1367] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.010s
I20240501 08:04:20.192081 17396 tablet_copy_source_session.cc:205] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:20.191924 18045 raft_consensus.cc:348] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.192857 18045 raft_consensus.cc:729] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:20.196398 18035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 068fca00dea44c6fbac5bc5365a66a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:20.193528 18045 consensus_queue.cc:260] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.200260 18045 ts_tablet_manager.cc:1398] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20240501 08:04:20.202257 17396 tablet_copy_service.cc:342] P 5655c2e9168d4eeaa212db8874c2efa5: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-e5f504d6ad6b4b77927fb216e12347e8 received from {username='slave'} at 127.0.0.1:44248
I20240501 08:04:20.202663 17396 tablet_copy_service.cc:433] P 5655c2e9168d4eeaa212db8874c2efa5: ending tablet copy session 010ae6a71a524121b75804307f9645f9-e5f504d6ad6b4b77927fb216e12347e8 on tablet e5f504d6ad6b4b77927fb216e12347e8 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:20.213407 18035 tablet_copy_client.cc:783] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:20.214205 18035 tablet_copy_client.cc:647] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:20.218029 18035 tablet_copy_client.cc:523] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:20.227100 18035 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:20.244196 18035 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:20.245090 18035 tablet_bootstrap.cc:492] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:20.245754 18035 ts_tablet_manager.cc:1367] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.007s
I20240501 08:04:20.248279 18035 raft_consensus.cc:348] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.249187 18035 raft_consensus.cc:729] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:20.249856 18035 consensus_queue.cc:260] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:20.251859 18035 ts_tablet_manager.cc:1398] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 08:04:20.254566 17396 tablet_copy_service.cc:342] P 5655c2e9168d4eeaa212db8874c2efa5: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-068fca00dea44c6fbac5bc5365a66a4b received from {username='slave'} at 127.0.0.1:44248
I20240501 08:04:20.256165 17396 tablet_copy_service.cc:433] P 5655c2e9168d4eeaa212db8874c2efa5: ending tablet copy session 010ae6a71a524121b75804307f9645f9-068fca00dea44c6fbac5bc5365a66a4b on tablet 068fca00dea44c6fbac5bc5365a66a4b with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:20.473018 17902 raft_consensus.cc:1203] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:20.490272 17902 raft_consensus.cc:1203] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:20.534812 17902 raft_consensus.cc:1203] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:20.684610 17902 raft_consensus.cc:1203] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:20.858052 18017 raft_consensus.cc:1050] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:20.859947 18017 consensus_queue.cc:237] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:20.865610 17902 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:20.865610 17385 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:20.865777 17307 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:20.867139 18060 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:20.868350 18063 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:20.869693 17939 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:04:20.888252 17902 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.884760 17939 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.894007 17383 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.904184 17133 catalog_manager.cc:5484] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.912165 17307 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.939772 17837 raft_consensus.cc:1050] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:20.941720 17837 consensus_queue.cc:237] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:20.959746 17902 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:20.961191 18031 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:20.965178 17307 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:20.965965 17232 consensus_queue.cc:237] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:20.968106 18031 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:04:20.972437 17230 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:20.974170 17903 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:20.974509 17837 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:20.975486 17310 raft_consensus.cc:1261] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:20.977362 18017 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:20.978969 18017 consensus_queue.cc:1035] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:20.988339 17902 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.986891 18063 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:20.999791 17903 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.000844 17310 raft_consensus.cc:2932] T e4b47c5f05da46bc9d9b5b4b202c1210 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.004088 18061 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.006810 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e4b47c5f05da46bc9d9b5b4b202c1210 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:21.007990 18062 raft_consensus.cc:1050] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:21.010396 18062 consensus_queue.cc:237] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:21.014736 17132 catalog_manager.cc:5484] T e4b47c5f05da46bc9d9b5b4b202c1210 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:04:21.018733 17233 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.021652 17135 catalog_manager.cc:5484] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.048818 17376 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b47c5f05da46bc9d9b5b4b202c1210 with delete_type TABLET_DATA_TOMBSTONED (TS 5655c2e9168d4eeaa212db8874c2efa5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37214
I20240501 08:04:21.063174 17305 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.071337 17307 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:21.071472 17902 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:21.073894 17230 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:21.074285 17837 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:21.075488 18079 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:21.082384 18079 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:04:21.088629 18080 tablet_replica.cc:330] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: stopping tablet replica
I20240501 08:04:21.093847 17382 consensus_queue.cc:237] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:21.094928 18080 raft_consensus.cc:2229] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:21.098536 17902 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:21.096436 18080 raft_consensus.cc:2258] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:21.099074 17307 raft_consensus.cc:1261] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:21.100284 18031 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:21.101962 18031 consensus_queue.cc:1035] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:21.112845 18080 ts_tablet_manager.cc:1875] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:21.114015 18079 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.117666 17837 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.117777 17902 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.118541 17305 raft_consensus.cc:2932] T 068fca00dea44c6fbac5bc5365a66a4b P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.125423 17310 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.133750 17902 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.135339 17135 catalog_manager.cc:5484] T 068fca00dea44c6fbac5bc5365a66a4b P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.135535 17233 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.143126 17122 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 068fca00dea44c6fbac5bc5365a66a4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:21.148443 18080 ts_tablet_manager.cc:1888] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:21.150779 18080 log.cc:1196] T e4b47c5f05da46bc9d9b5b4b202c1210 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root/wals/e4b47c5f05da46bc9d9b5b4b202c1210
I20240501 08:04:21.154160 17122 catalog_manager.cc:4830] TS 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): tablet e4b47c5f05da46bc9d9b5b4b202c1210 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:21.158349 17226 tablet_service.cc:1514] Processing DeleteTablet for tablet 068fca00dea44c6fbac5bc5365a66a4b with delete_type TABLET_DATA_TOMBSTONED (TS fa9d3bc74a68481e9de647223cb0110e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52016
I20240501 08:04:21.163386 17134 catalog_manager.cc:5484] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:04:21.168869 18084 tablet_replica.cc:330] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: stopping tablet replica
I20240501 08:04:21.184831 17837 raft_consensus.cc:1050] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:21.187057 17837 consensus_queue.cc:237] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:21.197022 18084 raft_consensus.cc:2229] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:21.197872 18084 raft_consensus.cc:2258] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:21.201113 18084 ts_tablet_manager.cc:1875] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:21.208930 17310 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:21.208966 17902 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:21.210700 17233 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:21.211081 18031 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:21.211941 17837 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:21.212743 18079 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:21.214146 18084 ts_tablet_manager.cc:1888] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:21.214551 18084 log.cc:1196] T 068fca00dea44c6fbac5bc5365a66a4b P fa9d3bc74a68481e9de647223cb0110e: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root/wals/068fca00dea44c6fbac5bc5365a66a4b
I20240501 08:04:21.216192 17121 catalog_manager.cc:4830] TS fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): tablet 068fca00dea44c6fbac5bc5365a66a4b (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:21.232260 17383 consensus_queue.cc:237] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:21.241400 18061 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.245018 17902 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:21.245347 17230 raft_consensus.cc:1261] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:21.244918 17903 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.246836 17837 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:21.247614 18079 consensus_queue.cc:1035] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:21.249830 17233 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.256500 18062 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.258415 17133 catalog_manager.cc:5484] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.267134 17307 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.270874 17122 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 66624ff54d8844e3bf88af88be895fd3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:21.283730 17902 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.293947 17385 consensus_queue.cc:237] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:21.299739 17233 raft_consensus.cc:2932] T 66624ff54d8844e3bf88af88be895fd3 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.305400 17132 catalog_manager.cc:5484] T 66624ff54d8844e3bf88af88be895fd3 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.311060 17902 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:21.315403 18062 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:21.319916 17295 tablet_service.cc:1514] Processing DeleteTablet for tablet 66624ff54d8844e3bf88af88be895fd3 with delete_type TABLET_DATA_TOMBSTONED (TS cdccadabd9154ea69e15405d75301fc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55168
I20240501 08:04:21.323132 18062 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.326922 17233 raft_consensus.cc:1261] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:21.333312 18079 consensus_queue.cc:1035] T e5f504d6ad6b4b77927fb216e12347e8 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:21.334836 17902 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.336395 17122 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e5f504d6ad6b4b77927fb216e12347e8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:21.340262 18089 tablet_replica.cc:330] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: stopping tablet replica
I20240501 08:04:21.339831 17230 raft_consensus.cc:2932] T e5f504d6ad6b4b77927fb216e12347e8 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:21.349162 18089 raft_consensus.cc:2229] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:21.355180 18089 raft_consensus.cc:2258] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:21.359493 18089 ts_tablet_manager.cc:1875] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:21.358796 17133 catalog_manager.cc:5484] T e5f504d6ad6b4b77927fb216e12347e8 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
W20240501 08:04:21.363289 17122 catalog_manager.cc:5007] ChangeConfig:REMOVE_PEER RPC for tablet e5f504d6ad6b4b77927fb216e12347e8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20240501 08:04:21.371358 17295 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f504d6ad6b4b77927fb216e12347e8 with delete_type TABLET_DATA_TOMBSTONED (TS cdccadabd9154ea69e15405d75301fc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55168
I20240501 08:04:21.373708 18089 ts_tablet_manager.cc:1888] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:21.374167 18089 log.cc:1196] T 66624ff54d8844e3bf88af88be895fd3 P cdccadabd9154ea69e15405d75301fc1: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root/wals/66624ff54d8844e3bf88af88be895fd3
I20240501 08:04:21.375375 17121 catalog_manager.cc:4830] TS cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): tablet 66624ff54d8844e3bf88af88be895fd3 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:21.375485 18089 tablet_replica.cc:330] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: stopping tablet replica
I20240501 08:04:21.376430 18089 raft_consensus.cc:2229] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:21.376943 18089 raft_consensus.cc:2258] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:21.378993 18089 ts_tablet_manager.cc:1875] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:21.389138 18089 ts_tablet_manager.cc:1888] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:21.389463 18089 log.cc:1196] T e5f504d6ad6b4b77927fb216e12347e8 P cdccadabd9154ea69e15405d75301fc1: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root/wals/e5f504d6ad6b4b77927fb216e12347e8
I20240501 08:04:21.390435 17121 catalog_manager.cc:4830] TS cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): tablet e5f504d6ad6b4b77927fb216e12347e8 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:22.436095 17230 consensus_queue.cc:237] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
W20240501 08:04:22.442143 17202 consensus_peers.cc:485] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 726057013c1e48f985e1fd55e502c0e0. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:22.443301 17383 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.443917 17307 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.444576 18092 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.445441 17939 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.452069 18092 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.457254 17307 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.467020 17382 consensus_queue.cc:237] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:22.461196 17383 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
W20240501 08:04:22.475869 17351 consensus_peers.cc:485] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 384fc32cbcd0455b9cce4f5e71f9a626. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:22.477897 17230 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.480947 18079 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.483857 17310 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.485189 18079 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.493130 17132 catalog_manager.cc:5484] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:22.507148 18090 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.512394 17230 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.531486 17310 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.533324 17230 consensus_queue.cc:237] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:22.549325 17133 catalog_manager.cc:5484] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:22.552609 17382 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.553021 17307 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240501 08:04:22.553189 17202 consensus_peers.cc:485] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3869c361a6d47679025e9c5e47b4706. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:22.554313 18063 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.555987 18063 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.576246 18092 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.577461 17307 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.581863 17383 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.590350 17307 consensus_queue.cc:237] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:22.594120 17133 catalog_manager.cc:5484] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:22.612417 17383 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.612605 17233 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:22.613528 17986 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240501 08:04:22.614367 17277 consensus_peers.cc:485] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a87f059d40b441d5ad84b2e410829e61. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:22.628201 17986 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:22.629983 17986 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.631335 17383 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.651253 17233 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.658308 17133 catalog_manager.cc:5484] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:22.894423 18110 ts_tablet_manager.cc:902] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:22.899859 18110 tablet_copy_client.cc:308] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.129:44541
I20240501 08:04:22.901563 17247 tablet_copy_service.cc:140] P fa9d3bc74a68481e9de647223cb0110e: Received BeginTabletCopySession request for tablet 726057013c1e48f985e1fd55e502c0e0 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:36634)
I20240501 08:04:22.902071 17247 tablet_copy_service.cc:161] P fa9d3bc74a68481e9de647223cb0110e: Beginning new tablet copy session on tablet 726057013c1e48f985e1fd55e502c0e0 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:36634: session id = 010ae6a71a524121b75804307f9645f9-726057013c1e48f985e1fd55e502c0e0
I20240501 08:04:22.908382 17247 tablet_copy_source_session.cc:205] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:22.911787 18110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726057013c1e48f985e1fd55e502c0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:22.912483 18115 ts_tablet_manager.cc:902] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
I20240501 08:04:22.927466 18110 tablet_copy_client.cc:783] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:22.928023 18110 tablet_copy_client.cc:647] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:22.931833 18110 tablet_copy_client.cc:523] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:22.940105 18110 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:22.950284 18115 tablet_copy_client.cc:308] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.131:43915
I20240501 08:04:22.952036 17396 tablet_copy_service.cc:140] P 5655c2e9168d4eeaa212db8874c2efa5: Received BeginTabletCopySession request for tablet 384fc32cbcd0455b9cce4f5e71f9a626 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:44248)
I20240501 08:04:22.952555 17396 tablet_copy_service.cc:161] P 5655c2e9168d4eeaa212db8874c2efa5: Beginning new tablet copy session on tablet 384fc32cbcd0455b9cce4f5e71f9a626 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:44248: session id = 010ae6a71a524121b75804307f9645f9-384fc32cbcd0455b9cce4f5e71f9a626
I20240501 08:04:22.961354 18110 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:22.961709 17396 tablet_copy_source_session.cc:205] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:22.962239 18110 tablet_bootstrap.cc:492] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:22.962931 18110 ts_tablet_manager.cc:1367] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.009s
I20240501 08:04:22.964947 18115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384fc32cbcd0455b9cce4f5e71f9a626. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:22.965176 18110 raft_consensus.cc:348] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:22.965865 18110 raft_consensus.cc:729] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:22.966434 18110 consensus_queue.cc:260] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:22.973523 18110 ts_tablet_manager.cc:1398] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20240501 08:04:22.975970 17247 tablet_copy_service.cc:342] P fa9d3bc74a68481e9de647223cb0110e: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-726057013c1e48f985e1fd55e502c0e0 received from {username='slave'} at 127.0.0.1:36634
I20240501 08:04:22.976365 17247 tablet_copy_service.cc:433] P fa9d3bc74a68481e9de647223cb0110e: ending tablet copy session 010ae6a71a524121b75804307f9645f9-726057013c1e48f985e1fd55e502c0e0 on tablet 726057013c1e48f985e1fd55e502c0e0 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:22.982023 18115 tablet_copy_client.cc:783] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:22.982578 18115 tablet_copy_client.cc:647] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:22.985832 18115 tablet_copy_client.cc:523] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:22.993631 18115 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:23.013312 18115 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:23.014194 18115 tablet_bootstrap.cc:492] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:23.014871 18115 ts_tablet_manager.cc:1367] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.008s
I20240501 08:04:23.017381 18115 raft_consensus.cc:348] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:23.017992 18115 raft_consensus.cc:729] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:23.018486 18115 consensus_queue.cc:260] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:23.020241 18115 ts_tablet_manager.cc:1398] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240501 08:04:23.021643 17396 tablet_copy_service.cc:342] P 5655c2e9168d4eeaa212db8874c2efa5: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-384fc32cbcd0455b9cce4f5e71f9a626 received from {username='slave'} at 127.0.0.1:44248
I20240501 08:04:23.021987 17396 tablet_copy_service.cc:433] P 5655c2e9168d4eeaa212db8874c2efa5: ending tablet copy session 010ae6a71a524121b75804307f9645f9-384fc32cbcd0455b9cce4f5e71f9a626 on tablet 384fc32cbcd0455b9cce4f5e71f9a626 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:23.040763 18115 ts_tablet_manager.cc:902] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
I20240501 08:04:23.051582 18115 tablet_copy_client.cc:308] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.129:44541
I20240501 08:04:23.053040 17247 tablet_copy_service.cc:140] P fa9d3bc74a68481e9de647223cb0110e: Received BeginTabletCopySession request for tablet b3869c361a6d47679025e9c5e47b4706 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:36634)
I20240501 08:04:23.053572 17247 tablet_copy_service.cc:161] P fa9d3bc74a68481e9de647223cb0110e: Beginning new tablet copy session on tablet b3869c361a6d47679025e9c5e47b4706 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:36634: session id = 010ae6a71a524121b75804307f9645f9-b3869c361a6d47679025e9c5e47b4706
I20240501 08:04:23.059929 17247 tablet_copy_source_session.cc:205] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:23.062572 18115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3869c361a6d47679025e9c5e47b4706. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:23.073734 18115 tablet_copy_client.cc:783] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:23.074225 18115 tablet_copy_client.cc:647] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:23.077356 18115 tablet_copy_client.cc:523] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:23.084815 18115 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:23.101563 18115 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:23.102447 18115 tablet_bootstrap.cc:492] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:23.103468 18115 ts_tablet_manager.cc:1367] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.019s user 0.020s sys 0.001s
I20240501 08:04:23.103927 18110 ts_tablet_manager.cc:902] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757)
I20240501 08:04:23.106081 18115 raft_consensus.cc:348] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:23.107136 18115 raft_consensus.cc:729] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:23.107764 18115 consensus_queue.cc:260] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:23.110193 18115 ts_tablet_manager.cc:1398] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:04:23.111586 17247 tablet_copy_service.cc:342] P fa9d3bc74a68481e9de647223cb0110e: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-b3869c361a6d47679025e9c5e47b4706 received from {username='slave'} at 127.0.0.1:36634
I20240501 08:04:23.111971 17247 tablet_copy_service.cc:433] P fa9d3bc74a68481e9de647223cb0110e: ending tablet copy session 010ae6a71a524121b75804307f9645f9-b3869c361a6d47679025e9c5e47b4706 on tablet b3869c361a6d47679025e9c5e47b4706 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:23.131866 18110 tablet_copy_client.cc:308] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.168.130:33757
I20240501 08:04:23.133215 17322 tablet_copy_service.cc:140] P cdccadabd9154ea69e15405d75301fc1: Received BeginTabletCopySession request for tablet a87f059d40b441d5ad84b2e410829e61 from peer 010ae6a71a524121b75804307f9645f9 ({username='slave'} at 127.0.0.1:50154)
I20240501 08:04:23.133672 17322 tablet_copy_service.cc:161] P cdccadabd9154ea69e15405d75301fc1: Beginning new tablet copy session on tablet a87f059d40b441d5ad84b2e410829e61 from peer 010ae6a71a524121b75804307f9645f9 at {username='slave'} at 127.0.0.1:50154: session id = 010ae6a71a524121b75804307f9645f9-a87f059d40b441d5ad84b2e410829e61
I20240501 08:04:23.139259 17322 tablet_copy_source_session.cc:205] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:04:23.141701 18110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87f059d40b441d5ad84b2e410829e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:04:23.153087 18110 tablet_copy_client.cc:783] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 0 data blocks...
I20240501 08:04:23.153594 18110 tablet_copy_client.cc:647] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: tablet copy: Starting download of 1 WAL segments...
I20240501 08:04:23.156468 18110 tablet_copy_client.cc:523] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:04:23.162601 18110 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: Bootstrap starting.
I20240501 08:04:23.179929 18110 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:04:23.180757 18110 tablet_bootstrap.cc:492] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: Bootstrap complete.
I20240501 08:04:23.181455 18110 ts_tablet_manager.cc:1367] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: Time spent bootstrapping tablet: real 0.019s user 0.017s sys 0.002s
I20240501 08:04:23.183465 18110 raft_consensus.cc:348] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:23.184195 18110 raft_consensus.cc:729] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 010ae6a71a524121b75804307f9645f9, State: Initialized, Role: LEARNER
I20240501 08:04:23.184792 18110 consensus_queue.cc:260] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:23.186825 18110 ts_tablet_manager.cc:1398] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20240501 08:04:23.188772 17322 tablet_copy_service.cc:342] P cdccadabd9154ea69e15405d75301fc1: Request end of tablet copy session 010ae6a71a524121b75804307f9645f9-a87f059d40b441d5ad84b2e410829e61 received from {username='slave'} at 127.0.0.1:50154
I20240501 08:04:23.189152 17322 tablet_copy_service.cc:433] P cdccadabd9154ea69e15405d75301fc1: ending tablet copy session 010ae6a71a524121b75804307f9645f9-a87f059d40b441d5ad84b2e410829e61 on tablet a87f059d40b441d5ad84b2e410829e61 with peer 010ae6a71a524121b75804307f9645f9
I20240501 08:04:23.274708 17903 raft_consensus.cc:1203] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:23.442399 17903 raft_consensus.cc:1203] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:23.459476 17903 raft_consensus.cc:1203] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:23.682145 17903 raft_consensus.cc:1203] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 08:04:23.772505 18106 raft_consensus.cc:1050] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:23.774979 18106 consensus_queue.cc:237] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:23.784310 17903 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:23.785069 17305 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:23.785928 17385 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:23.785785 18106 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20240501 08:04:23.789794 18063 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:23.791819 17939 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:23.819269 18063 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.820585 17305 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.838323 18079 raft_consensus.cc:1050] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:23.839843 17134 catalog_manager.cc:5484] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.840492 18079 consensus_queue.cc:237] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:23.855219 17382 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.858912 17903 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.864233 17902 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:23.865758 18062 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:23.871238 17305 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:23.876449 17232 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:23.877067 18133 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20240501 08:04:23.886885 18133 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:23.909160 18133 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.910900 17305 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.927337 17230 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.927774 17903 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.950598 17132 catalog_manager.cc:5484] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.967398 17230 consensus_queue.cc:237] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:23.969691 17382 consensus_queue.cc:237] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:23.972786 17902 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:23.974207 17902 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:23.975008 17233 raft_consensus.cc:1261] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:23.974858 18106 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:23.975978 18146 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:23.978533 17305 raft_consensus.cc:1261] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:23.979198 18079 consensus_queue.cc:1035] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:23.979425 17939 raft_consensus.cc:1050] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:23.979974 18142 consensus_queue.cc:1035] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:23.985162 17939 consensus_queue.cc:237] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:23.986347 18133 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.991549 17311 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.997808 17901 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.999042 17899 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:24.000451 18142 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:24.001528 17899 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:23.987782 18106 raft_consensus.cc:2932] T 726057013c1e48f985e1fd55e502c0e0 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.002831 17383 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:24.009416 18142 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:24.016927 17305 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:24.018203 18136 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:24.031497 17122 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 384fc32cbcd0455b9cce4f5e71f9a626 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:24.033035 17233 raft_consensus.cc:2932] T 384fc32cbcd0455b9cce4f5e71f9a626 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.042249 17134 catalog_manager.cc:5484] T 384fc32cbcd0455b9cce4f5e71f9a626 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index 3 to 4, VOTER cdccadabd9154ea69e15405d75301fc1 (127.15.168.130) evicted. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:04:24.015661 17135 catalog_manager.cc:5484] T 726057013c1e48f985e1fd55e502c0e0 P 010ae6a71a524121b75804307f9645f9 reported cstate change: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.053071 18136 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.054534 17311 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.063230 17295 tablet_service.cc:1514] Processing DeleteTablet for tablet 384fc32cbcd0455b9cce4f5e71f9a626 with delete_type TABLET_DATA_TOMBSTONED (TS cdccadabd9154ea69e15405d75301fc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55168
I20240501 08:04:24.070371 17376 tablet_service.cc:1514] Processing DeleteTablet for tablet 726057013c1e48f985e1fd55e502c0e0 with delete_type TABLET_DATA_TOMBSTONED (TS 5655c2e9168d4eeaa212db8874c2efa5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37214
I20240501 08:04:24.077487 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 726057013c1e48f985e1fd55e502c0e0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:24.093412 18157 tablet_replica.cc:330] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: stopping tablet replica
I20240501 08:04:24.094722 17899 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.097735 18157 raft_consensus.cc:2229] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:24.098497 18157 raft_consensus.cc:2258] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:24.093137 17133 catalog_manager.cc:5484] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.100071 18156 tablet_replica.cc:330] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: stopping tablet replica
I20240501 08:04:24.101119 18156 raft_consensus.cc:2229] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:24.101852 18156 raft_consensus.cc:2258] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:24.102015 18157 ts_tablet_manager.cc:1875] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:24.104024 17383 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.114087 18156 ts_tablet_manager.cc:1875] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:24.126791 17230 consensus_queue.cc:237] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:24.129386 18156 ts_tablet_manager.cc:1888] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:24.129918 18156 log.cc:1196] T 384fc32cbcd0455b9cce4f5e71f9a626 P cdccadabd9154ea69e15405d75301fc1: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-1-root/wals/384fc32cbcd0455b9cce4f5e71f9a626
I20240501 08:04:24.133908 17121 catalog_manager.cc:4830] TS cdccadabd9154ea69e15405d75301fc1 (127.15.168.130:33757): tablet 384fc32cbcd0455b9cce4f5e71f9a626 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:24.137576 17311 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:24.139326 18136 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:24.142678 17900 raft_consensus.cc:1261] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:24.147387 18136 consensus_queue.cc:1035] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:24.158371 17939 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.162690 18157 ts_tablet_manager.cc:1888] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:24.163596 18157 log.cc:1196] T 726057013c1e48f985e1fd55e502c0e0 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root/wals/726057013c1e48f985e1fd55e502c0e0
I20240501 08:04:24.165380 17122 catalog_manager.cc:4830] TS 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): tablet 726057013c1e48f985e1fd55e502c0e0 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:24.166929 17311 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.168285 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b3869c361a6d47679025e9c5e47b4706 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:24.177119 17899 raft_consensus.cc:2932] T b3869c361a6d47679025e9c5e47b4706 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.180955 17135 catalog_manager.cc:5484] T b3869c361a6d47679025e9c5e47b4706 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 3 to 4, VOTER 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131) evicted. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.195452 18134 raft_consensus.cc:1050] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1: attempting to promote NON_VOTER 010ae6a71a524121b75804307f9645f9 to VOTER
I20240501 08:04:24.197490 18134 consensus_queue.cc:237] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:24.201704 17376 tablet_service.cc:1514] Processing DeleteTablet for tablet b3869c361a6d47679025e9c5e47b4706 with delete_type TABLET_DATA_TOMBSTONED (TS 5655c2e9168d4eeaa212db8874c2efa5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37214
I20240501 08:04:24.202667 18157 tablet_replica.cc:330] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: stopping tablet replica
I20240501 08:04:24.203680 18157 raft_consensus.cc:2229] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:24.204306 18157 raft_consensus.cc:2258] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:24.208499 18157 ts_tablet_manager.cc:1875] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:24.214061 17384 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:24.214659 17899 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 LEARNER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:24.216497 18134 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:24.219985 17233 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:04:24.220834 18107 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240501 08:04:24.223812 18161 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 08:04:24.239495 18157 ts_tablet_manager.cc:1888] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:24.239961 18157 log.cc:1196] T b3869c361a6d47679025e9c5e47b4706 P 5655c2e9168d4eeaa212db8874c2efa5: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-2-root/wals/b3869c361a6d47679025e9c5e47b4706
I20240501 08:04:24.242270 17122 catalog_manager.cc:4830] TS 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915): tablet b3869c361a6d47679025e9c5e47b4706 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:24.241204 18158 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.242398 17233 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.249099 17384 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.262957 17899 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.265195 17133 catalog_manager.cc:5484] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index 2 to 3, 010ae6a71a524121b75804307f9645f9 (127.15.168.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.283365 17311 consensus_queue.cc:237] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } }
I20240501 08:04:24.288013 17384 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:24.289090 17899 raft_consensus.cc:1261] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Refusing update from remote peer cdccadabd9154ea69e15405d75301fc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:04:24.289367 18160 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240501 08:04:24.291613 18158 consensus_queue.cc:1035] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:04:24.295646 18161 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.299099 17384 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.303582 17121 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a87f059d40b441d5ad84b2e410829e61 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:04:24.311053 17899 raft_consensus.cc:2932] T a87f059d40b441d5ad84b2e410829e61 P 010ae6a71a524121b75804307f9645f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } } }
I20240501 08:04:24.313607 17134 catalog_manager.cc:5484] T a87f059d40b441d5ad84b2e410829e61 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index 3 to 4, VOTER fa9d3bc74a68481e9de647223cb0110e (127.15.168.129) evicted. New cstate: current_term: 1 leader_uuid: "cdccadabd9154ea69e15405d75301fc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:04:24.327824 17226 tablet_service.cc:1514] Processing DeleteTablet for tablet a87f059d40b441d5ad84b2e410829e61 with delete_type TABLET_DATA_TOMBSTONED (TS fa9d3bc74a68481e9de647223cb0110e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52016
I20240501 08:04:24.337852 18165 tablet_replica.cc:330] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: stopping tablet replica
I20240501 08:04:24.339469 18165 raft_consensus.cc:2229] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:04:24.339946 18165 raft_consensus.cc:2258] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:04:24.342196 18165 ts_tablet_manager.cc:1875] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:04:24.354498 18165 ts_tablet_manager.cc:1888] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 08:04:24.354955 18165 log.cc:1196] T a87f059d40b441d5ad84b2e410829e61 P fa9d3bc74a68481e9de647223cb0110e: Deleting WAL directory at /tmp/dist-test-taskJM4W8L/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714550584420650-16034-0/minicluster-data/ts-0-root/wals/a87f059d40b441d5ad84b2e410829e61
I20240501 08:04:24.356971 17121 catalog_manager.cc:4830] TS fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541): tablet a87f059d40b441d5ad84b2e410829e61 (table test-workload [id=92a9e0c772584e2e8ca694d0cf557b87]) successfully deleted
I20240501 08:04:25.388628 17383 consensus_queue.cc:237] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
W20240501 08:04:25.395650 17351 consensus_peers.cc:485] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d636a072366c4776aecc4b164749d410. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:25.396875 17305 raft_consensus.cc:1261] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.396005 17230 raft_consensus.cc:1261] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Refusing update from remote peer 5655c2e9168d4eeaa212db8874c2efa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.398205 18133 consensus_queue.cc:1035] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:04:25.399150 18168 consensus_queue.cc:1035] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:25.412609 18153 raft_consensus.cc:2932] T d636a072366c4776aecc4b164749d410 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.414232 17230 raft_consensus.cc:2932] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.415982 17305 raft_consensus.cc:2932] T d636a072366c4776aecc4b164749d410 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.428740 17230 consensus_queue.cc:237] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:25.430851 17132 catalog_manager.cc:5484] T d636a072366c4776aecc4b164749d410 P fa9d3bc74a68481e9de647223cb0110e reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "5655c2e9168d4eeaa212db8874c2efa5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } attrs { replace: true } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
W20240501 08:04:25.437172 17202 consensus_peers.cc:485] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3abec6d380a4a5fa72fd34fe37bbd92. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:25.438426 17383 raft_consensus.cc:1261] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.439766 18154 consensus_queue.cc:1035] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:04:25.435695 17305 raft_consensus.cc:1261] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.445473 17939 consensus_queue.cc:1035] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240501 08:04:25.449388 18167 raft_consensus.cc:2932] T d3abec6d380a4a5fa72fd34fe37bbd92 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.458107 17383 raft_consensus.cc:2932] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.466910 17230 consensus_queue.cc:237] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:25.486649 17305 raft_consensus.cc:2932] T d3abec6d380a4a5fa72fd34fe37bbd92 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.494454 17383 raft_consensus.cc:1261] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240501 08:04:25.494580 17202 consensus_peers.cc:485] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5e141ff8f064eb480d6e2174c541e42. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:25.495303 17311 raft_consensus.cc:1261] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.496526 18167 consensus_queue.cc:1035] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:25.497483 17939 consensus_queue.cc:1035] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:25.506379 18167 raft_consensus.cc:2932] T e5e141ff8f064eb480d6e2174c541e42 P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.520653 17135 catalog_manager.cc:5484] T d3abec6d380a4a5fa72fd34fe37bbd92 P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.525590 17305 raft_consensus.cc:2932] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.526681 17230 consensus_queue.cc:237] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } }
I20240501 08:04:25.530872 17383 raft_consensus.cc:2932] T e5e141ff8f064eb480d6e2174c541e42 P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.549463 17135 catalog_manager.cc:5484] T e5e141ff8f064eb480d6e2174c541e42 P cdccadabd9154ea69e15405d75301fc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } attrs { replace: true } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
W20240501 08:04:25.564708 17202 consensus_peers.cc:485] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e -> Peer 010ae6a71a524121b75804307f9645f9 (127.15.168.132:45715): Couldn't send request to peer 010ae6a71a524121b75804307f9645f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f12aee8e217041f19aa8102e5fe1507f. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:04:25.565718 17305 raft_consensus.cc:1261] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.565780 17383 raft_consensus.cc:1261] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Refusing update from remote peer fa9d3bc74a68481e9de647223cb0110e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:04:25.567114 17939 consensus_queue.cc:1035] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:25.567994 18167 consensus_queue.cc:1035] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 08:04:25.580315 18167 raft_consensus.cc:2932] T f12aee8e217041f19aa8102e5fe1507f P fa9d3bc74a68481e9de647223cb0110e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.581877 17383 raft_consensus.cc:2932] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.593958 17305 raft_consensus.cc:2932] T f12aee8e217041f19aa8102e5fe1507f P cdccadabd9154ea69e15405d75301fc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.595602 17132 catalog_manager.cc:5484] T f12aee8e217041f19aa8102e5fe1507f P 5655c2e9168d4eeaa212db8874c2efa5 reported cstate change: config changed from index -1 to 2, NON_VOTER 010ae6a71a524121b75804307f9645f9 (127.15.168.132) added. New cstate: current_term: 1 leader_uuid: "fa9d3bc74a68481e9de647223cb0110e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa9d3bc74a68481e9de647223cb0110e" member_type: VOTER last_known_addr { host: "127.15.168.129" port: 44541 } } peers { permanent_uuid: "cdccadabd9154ea69e15405d75301fc1" member_type: VOTER last_known_addr { host: "127.15.168.130" port: 33757 } } peers { permanent_uuid: "5655c2e9168d4eeaa212db8874c2efa5" member_type: VOTER last_known_addr { host: "127.15.168.131" port: 43915 } attrs { replace: true } } peers { permanent_uuid: "010ae6a71a524121b75804307f9645f9" member_type: NON_VOTER last_known_addr { host: "127.15.168.132" port: 45715 } attrs { promote: true } } }
I20240501 08:04:25.827469 18187 ts_tablet_manager.cc:902] T d636a072366c4776aecc4b164749d410 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer 5655c2e9168d4eeaa212db8874c2efa5 (127.15.168.131:43915)
F20240501 08:04:25.828522 18187 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (init.queue_time_us, init.queue_time_us) (init.run_cpu_time_us, init.run_cpu_time_us) (init.run_wall_time_us, init.run_wall_time_us) (leader-initialization.queue_time_us, leader-initialization.queue_time_us) (leader-initialization.run_cpu_time_us, leader-initialization.run_cpu_time_us) (leader-initialization.run_wall_time_us, leader-initialization.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-068fca00dea44c6fbac5bc5365a66a4b.queue_time_us, tablet-download-pool-068fca00dea44c6fbac5bc5365a66a4b.queue_time_us) (tablet-download-pool-068fca00dea44c6fbac5bc5365a66a4b.run_cpu_time_us, tablet-download-pool-068fca00dea44c6fbac5bc5365a66a4b.run_cpu_time_us) (tablet-download-pool-068fca00dea44c6fbac5bc5365a66a4b.run_wall_time_us, tablet-download-pool-068fca00dea44c6fbac5bc5365a66a4b.run_wall_time_us) (tablet-download-pool-1189be88e7b341918c1fa148b2d77030.queue_time_us, tablet-download-pool-1189be88e7b341918c1fa148b2d77030.queue_time_us) (tablet-download-pool-1189be88e7b341918c1fa148b2d77030.run_cpu_time_us, tablet-download-pool-1189be88e7b341918c1fa148b2d77030.run_cpu_time_us) (tablet-download-pool-1189be88e7b341918c1fa148b2d77030.run_wall_time_us, tablet-download-pool-1189be88e7b341918c1fa148b2d77030.run_wall_time_us) (tablet-download-pool-2250a60c258544668fa164fcf11634e4.queue_time_us, tablet-download-pool-2250a60c258544668fa164fcf11634e4.queue_time_us) (tablet-download-pool-2250a60c258544668fa164fcf11634e4.run_cpu_time_us, tablet-download-pool-2250a60c258544668fa164fcf11634e4.run_cpu_time_us) (tablet-download-pool-2250a60c258544668fa164fcf11634e4.run_wall_time_us, tablet-download-pool-2250a60c258544668fa164fcf11634e4.run_wall_time_us) (tablet-download-pool-384fc32cbcd0455b9cce4f5e71f9a626.queue_time_us, tablet-download-pool-384fc32cbcd0455b9cce4f5e71f9a626.queue_time_us) (tablet-download-pool-384fc32cbcd0455b9cce4f5e71f9a626.run_cpu_time_us, tablet-download-pool-384fc32cbcd0455b9cce4f5e71f9a626.run_cpu_time_us) (tablet-download-pool-384fc32cbcd0455b9cce4f5e71f9a626.run_wall_time_us, tablet-download-pool-384fc32cbcd0455b9cce4f5e71f9a626.run_wall_time_us) (tablet-download-pool-3e00624b7db1409890298f5a6d88a575.queue_time_us, tablet-download-pool-3e00624b7db1409890298f5a6d88a575.queue_time_us) (tablet-download-pool-3e00624b7db1409890298f5a6d88a575.run_cpu_time_us, tablet-download-pool-3e00624b7db1409890298f5a6d88a575.run_cpu_time_us) (tablet-download-pool-3e00624b7db1409890298f5a6d88a575.run_wall_time_us, tablet-download-pool-3e00624b7db1409890298f5a6d88a575.run_wall_time_us) (tablet-download-pool-66624ff54d8844e3bf88af88be895fd3.queue_time_us, tablet-download-pool-66624ff54d8844e3bf88af88be895fd3.queue_time_us) (tablet-download-pool-66624ff54d8844e3bf88af88be895fd3.run_cpu_time_us, tablet-download-pool-66624ff54d8844e3bf88af88be895fd3.run_cpu_time_us) (tablet-download-pool-66624ff54d8844e3bf88af88be895fd3.run_wall_time_us, tablet-download-pool-66624ff54d8844e3bf88af88be895fd3.run_wall_time_us) (tablet-download-pool-726057013c1e48f985e1fd55e502c0e0.queue_time_us, tablet-download-pool-726057013c1e48f985e1fd55e502c0e0.queue_time_us) (tablet-download-pool-726057013c1e48f985e1fd55e502c0e0.run_cpu_time_us, tablet-download-pool-726057013c1e48f985e1fd55e502c0e0.run_cpu_time_us) (tablet-download-pool-726057013c1e48f985e1fd55e502c0e0.run_wall_time_us, tablet-download-pool-726057013c1e48f985e1fd55e502c0e0.run_wall_time_us) (tablet-download-pool-9cd19ca98cfd4974ad3fca19eb088cc0.queue_time_us, tablet-download-pool-9cd19ca98cfd4974ad3fca19eb088cc0.queue_time_us) (tablet-download-pool-9cd19ca98cfd4974ad3fca19eb088cc0.run_cpu_time_us, tablet-download-pool-9cd19ca98cfd4974ad3fca19eb088cc0.run_cpu_time_us) (tablet-download-pool-9cd19ca98cfd4974ad3fca19eb088cc0.run_wall_time_us, tablet-download-pool-9cd19ca98cfd4974ad3fca19eb088cc0.run_wall_time_us) (tablet-download-pool-a87f059d40b441d5ad84b2e410829e61.queue_time_us, tablet-download-pool-a87f059d40b441d5ad84b2e410829e61.queue_time_us) (tablet-download-pool-a87f059d40b441d5ad84b2e410829e61.run_cpu_time_us, tablet-download-pool-a87f059d40b441d5ad84b2e410829e61.run_cpu_time_us) (tablet-download-pool-a87f059d40b441d5ad84b2e410829e61.run_wall_time_us, tablet-download-pool-a87f059d40b441d5ad84b2e410829e61.run_wall_time_us) (tablet-download-pool-b3869c361a6d47679025e9c5e47b4706.queue_time_us, tablet-download-pool-b3869c361a6d47679025e9c5e47b4706.queue_time_us) (tablet-download-pool-b3869c361a6d47679025e9c5e47b4706.run_cpu_time_us, tablet-download-pool-b3869c361a6d47679025e9c5e47b4706.run_cpu_time_us) (tablet-download-pool-b3869c361a6d47679025e9c5e47b4706.run_wall_time_us, tablet-download-pool-b3869c361a6d47679025e9c5e47b4706.run_wall_time_us) (tablet-download-pool-d636a072366c4776aecc4b164749d410.queue_time_us, tablet-download-pool-d636a072366c4776aecc4b164749d410.queue_time_us) (tablet-download-pool-e4b47c5f05da46bc9d9b5b4b202c1210.queue_time_us, tablet-download-pool-e4b47c5f05da46bc9d9b5b4b202c1210.queue_time_us) (tablet-download-pool-e4b47c5f05da46bc9d9b5b4b202c1210.run_cpu_time_us, tablet-download-pool-e4b47c5f05da46bc9d9b5b4b202c1210.run_cpu_time_us) (tablet-download-pool-e4b47c5f05da46bc9d9b5b4b202c1210.run_wall_time_us, tablet-download-pool-e4b47c5f05da46bc9d9b5b4b202c1210.run_wall_time_us) (tablet-download-pool-e5f504d6ad6b4b77927fb216e12347e8.queue_time_us, tablet-download-pool-e5f504d6ad6b4b77927fb216e12347e8.queue_time_us) (tablet-download-pool-e5f504d6ad6b4b77927fb216e12347e8.run_cpu_time_us, tablet-download-pool-e5f504d6ad6b4b77927fb216e12347e8.run_cpu_time_us) (tablet-download-pool-e5f504d6ad6b4b77927fb216e12347e8.run_wall_time_us, tablet-download-pool-e5f504d6ad6b4b77927fb216e12347e8.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
*** Aborted at 1714550665 (unix time) try "date -d @1714550665" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800003ea2) received by PID 16034 (TID 0x7f7be1bd6700) from PID 16034; stack trace: ***
@ 0x2b0110 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b25f3 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
I20240501 08:04:25.890590 18189 ts_tablet_manager.cc:902] T e5e141ff8f064eb480d6e2174c541e42 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
@ 0x7f7c95fa6980 (unknown) at ??:0
@ 0x7f7c8f51afb7 gsignal at ??:0
@ 0x7f7c8f51c921 abort at ??:0
@ 0x2b6106 abort at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f7c92310da4 google::LogMessage::SendToLog() at ??:0
@ 0x7f7c92311950 google::LogMessage::Flush() at ??:0
I20240501 08:04:26.002148 18192 ts_tablet_manager.cc:902] T d3abec6d380a4a5fa72fd34fe37bbd92 P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
@ 0x7f7c92316a8b google::LogMessageFatal::~LogMessageFatal() at ??:0
I20240501 08:04:26.022881 18195 ts_tablet_manager.cc:902] T f12aee8e217041f19aa8102e5fe1507f P 010ae6a71a524121b75804307f9645f9: Initiating tablet copy from peer fa9d3bc74a68481e9de647223cb0110e (127.15.168.129:44541)
@ 0x7f7c92c9e3c7 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f7c92c7494f kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f7c92c720a5 kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f7c9aa92422 kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f7c9aa98c89 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f7c9ab2e4ef kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f7c9ab5f3cf kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f7c9ab451ce kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
@ 0x7f7c9ab4517a _ZNSt3__18__invokeIRZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBENS_8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEE4$_12JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSH_DpOSI_ at ??:0
@ 0x7f7c9ab4510a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7f7c9ab450d2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7f7c9ab442fe std::__1::__function::__func<>::operator()() at ??:0
@ 0x7f7c9beeada5 std::__1::__function::__value_func<>::operator()() at ??:0
@ 0x7f7c9beeabda std::__1::function<>::operator()() at ??:0
@ 0x7f7c92c76237 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7c92c79692 kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7c92c7964a _ZNSt3__18__invokeIRZN4kudu10ThreadPool12CreateThreadEvE3$_2JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS5_DpOS6_ at ??:0
@ 0x7f7c92c795da std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7f7c92c795a2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7f7c92c7889e std::__1::__function::__func<>::operator()() at ??:0
@ 0x7f7c9beeada5 std::__1::__function::__value_func<>::operator()() at ??:0
@ 0x7f7c9beeabda std::__1::function<>::operator()() at ??:0
@ 0x7f7c92c58937 kudu::Thread::SuperviseThread() at ??:0